[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:13:10 dist-test-slave-04bb krb5kdc[7448](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:13:10 dist-test-slave-04bb krb5kdc[7448](info): set up 2 sockets
Aug 11 05:13:10 dist-test-slave-04bb krb5kdc[7448](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 05:13:11.955603 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:13:12 dist-test-slave-04bb krb5kdc[7448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:13:12.088160 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:42357
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:42357
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:12.209167 7468 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:12.209450 7468 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:12.209519 7468 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:12.213531 7468 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:12.213609 7468 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:12.213649 7468 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:12.213708 7468 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:12.213747 7468 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:12.218818 7468 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:42357
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:42357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7468
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:12.219877 7468 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:12.220956 7468 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:12.230991 7479 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:12.231060 7477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:12.231045 7468 server_base.cc:1034] running on GCE node
W20260811 05:13:12.231314 7476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:12.231878 7468 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:12.233053 7468 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:12.234246 7468 hybrid_clock.cc:648] HybridClock initialized: now 1786425192234242 us; error 68 us; skew 500 ppm
Aug 11 05:13:12 dist-test-slave-04bb krb5kdc[7448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425192, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:12.237056 7468 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:13:12.237986 7468 webserver.cc:460] Webserver started at http://127.7.67.190:45983/ using document root <none> and password file <none>
I20260811 05:13:12.238552 7468 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:12.238642 7468 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:12.238880 7468 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:12.240991 7468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "93b7a05e85fd497096ed451de228d7ac"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-04bb"
server_key: "0e9375bf056eb7f8f1366e0c45e75df1"
server_key_iv: "be7c2cc095b4e040169583d848e1fc71"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.241550 7468 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "93b7a05e85fd497096ed451de228d7ac"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-04bb"
server_key: "0e9375bf056eb7f8f1366e0c45e75df1"
server_key_iv: "be7c2cc095b4e040169583d848e1fc71"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.245327 7468 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 05:13:12.247771 7488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.249047 7468 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:13:12.249200 7468 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "93b7a05e85fd497096ed451de228d7ac"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-04bb"
server_key: "0e9375bf056eb7f8f1366e0c45e75df1"
server_key_iv: "be7c2cc095b4e040169583d848e1fc71"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.249310 7468 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:12.266629 7468 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:12.270043 7468 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:12.270255 7468 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:12.277868 7468 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:42357
I20260811 05:13:12.277895 7568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:42357 every 8 connection(s)
I20260811 05:13:12.278855 7468 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:12.281775 7569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:12.284739 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 7468
I20260811 05:13:12.284893 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:13:12.286152 7438 external_mini_cluster.cc:1428] Setting key 24b95f952f449dd2db1c44266fcd77db
I20260811 05:13:12.287629 7569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93b7a05e85fd497096ed451de228d7ac: Bootstrap starting.
I20260811 05:13:12.290537 7569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93b7a05e85fd497096ed451de228d7ac: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:12.291347 7569 log.cc:824] T 00000000000000000000000000000000 P 93b7a05e85fd497096ed451de228d7ac: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:12.293320 7569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93b7a05e85fd497096ed451de228d7ac: No bootstrap required, opened a new log
I20260811 05:13:12.297215 7569 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93b7a05e85fd497096ed451de228d7ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93b7a05e85fd497096ed451de228d7ac" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42357 } }
I20260811 05:13:12.297447 7569 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93b7a05e85fd497096ed451de228d7ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:12.297528 7569 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93b7a05e85fd497096ed451de228d7ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93b7a05e85fd497096ed451de228d7ac, State: Initialized, Role: FOLLOWER
I20260811 05:13:12.297950 7569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93b7a05e85fd497096ed451de228d7ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93b7a05e85fd497096ed451de228d7ac" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42357 } }
I20260811 05:13:12.298115 7569 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93b7a05e85fd497096ed451de228d7ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:12.298206 7569 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93b7a05e85fd497096ed451de228d7ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:12.298327 7569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93b7a05e85fd497096ed451de228d7ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:12.299172 7569 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93b7a05e85fd497096ed451de228d7ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93b7a05e85fd497096ed451de228d7ac" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42357 } }
I20260811 05:13:12.299507 7569 leader_election.cc:302] T 00000000000000000000000000000000 P 93b7a05e85fd497096ed451de228d7ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93b7a05e85fd497096ed451de228d7ac; no voters:
I20260811 05:13:12.299853 7569 leader_election.cc:288] T 00000000000000000000000000000000 P 93b7a05e85fd497096ed451de228d7ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:12.299958 7575 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93b7a05e85fd497096ed451de228d7ac [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:12.300186 7575 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93b7a05e85fd497096ed451de228d7ac [term 1 LEADER]: Becoming Leader. State: Replica: 93b7a05e85fd497096ed451de228d7ac, State: Running, Role: LEADER
I20260811 05:13:12.300485 7575 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93b7a05e85fd497096ed451de228d7ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93b7a05e85fd497096ed451de228d7ac" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42357 } }
I20260811 05:13:12.301162 7569 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93b7a05e85fd497096ed451de228d7ac [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:12.301676 7578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93b7a05e85fd497096ed451de228d7ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93b7a05e85fd497096ed451de228d7ac. Latest consensus state: current_term: 1 leader_uuid: "93b7a05e85fd497096ed451de228d7ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93b7a05e85fd497096ed451de228d7ac" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42357 } } }
I20260811 05:13:12.301795 7578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93b7a05e85fd497096ed451de228d7ac [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:12.302513 7577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93b7a05e85fd497096ed451de228d7ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93b7a05e85fd497096ed451de228d7ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93b7a05e85fd497096ed451de228d7ac" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42357 } } }
I20260811 05:13:12.302616 7577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93b7a05e85fd497096ed451de228d7ac [sys.catalog]: This master's current role is: LEADER
Aug 11 05:13:12 dist-test-slave-04bb krb5kdc[7448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
W20260811 05:13:12.306624 7597 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 93b7a05e85fd497096ed451de228d7ac: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:13:12.306937 7597 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:13:12.307001 7584 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:12.307983 7584 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:12.312214 7571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.289381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60886 (local address 127.7.67.190:42357)
0811 05:13:12.289806 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:12.289822 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:12.290973 (+ 1151us) server_negotiation.cc:407] Connection header received
0811 05:13:12.294480 (+ 3507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:12.294496 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:12.294788 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:12.295099 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:12.297288 (+ 2189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.298622 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.299902 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.300615 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.300785 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.305770 (+ 4985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:12.305776 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:12.305788 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:12.305822 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:12.307719 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.308397 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.308403 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.308407 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.308519 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.308942 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.308945 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.308947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.309341 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:12.309547 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:12.310175 (+ 628us) server_negotiation.cc:299] Negotiation successful
0811 05:13:12.310494 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":133,"threads_started":1}
I20260811 05:13:12.314033 7584 catalog_manager.cc:1347] Generated new cluster ID: 8e4d5d27e09a4fc085a40a51af7b4b54
I20260811 05:13:12.314093 7584 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:12.320361 7584 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:12.321509 7584 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:12.327400 7584 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93b7a05e85fd497096ed451de228d7ac: Generated new TSK 0
I20260811 05:13:12.328037 7584 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
I20260811 05:13:12.365723 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:42357
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:12.482470 7606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:12.482744 7606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:12.482821 7606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:12.486778 7606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:12.486858 7606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:12.486944 7606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:13:12.492061 7606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:42357
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:12.493068 7606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:12.494066 7606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:12.503722 7618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:12.503681 7616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:12.503677 7615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:12.503993 7606 server_base.cc:1034] running on GCE node
I20260811 05:13:12.504467 7606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:12.505142 7606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:12.506376 7606 hybrid_clock.cc:648] HybridClock initialized: now 1786425192506344 us; error 48 us; skew 500 ppm
Aug 11 05:13:12 dist-test-slave-04bb krb5kdc[7448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425192, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:12.509428 7606 init.cc:377] Logged in from keytab as kudu/127.7.67.129@KRBTEST.COM (short username kudu)
I20260811 05:13:12.510262 7606 webserver.cc:460] Webserver started at http://127.7.67.129:43785/ using document root <none> and password file <none>
I20260811 05:13:12.510813 7606 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:12.510905 7606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:12.511298 7606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:12.513621 7606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "683a8de6d111487ab976faf0c6f7d35d"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-04bb"
server_key: "4ba924ce2a28f7682eeeec6b02bb539c"
server_key_iv: "960ef170a55a1bdfae1882f905871aa9"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.514246 7606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "683a8de6d111487ab976faf0c6f7d35d"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-04bb"
server_key: "4ba924ce2a28f7682eeeec6b02bb539c"
server_key_iv: "960ef170a55a1bdfae1882f905871aa9"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.518504 7606 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 05:13:12.521111 7628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.522852 7606 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:13:12.523015 7606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "683a8de6d111487ab976faf0c6f7d35d"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-04bb"
server_key: "4ba924ce2a28f7682eeeec6b02bb539c"
server_key_iv: "960ef170a55a1bdfae1882f905871aa9"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.523144 7606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:12.535053 7606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:12.538735 7606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:12.538960 7606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:12.539610 7606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:12.540555 7606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:12.540645 7606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.540724 7606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:12.540781 7606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.551122 7606 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:45081
I20260811 05:13:12.551144 7804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:45081 every 8 connection(s)
I20260811 05:13:12.552064 7606 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:12.552382 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 7606
I20260811 05:13:12.552467 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:13:12.552800 7438 external_mini_cluster.cc:1428] Setting key 61830ee40002dd4204c4c641289179b6
Aug 11 05:13:12 dist-test-slave-04bb krb5kdc[7448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425192, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.130@KRBTEST.COM" created.
I20260811 05:13:12.567737 7571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.554362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:45937 (local address 127.7.67.190:42357)
0811 05:13:12.554514 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:12.554519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:12.555575 (+ 1056us) server_negotiation.cc:407] Connection header received
0811 05:13:12.557311 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:12.557314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:12.557376 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:12.557482 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:12.558635 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.559491 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.560880 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.561319 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.561461 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.563875 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:12.563883 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:12.563885 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:12.563919 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:12.565645 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.566192 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.566196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.566198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.566291 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.566639 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.566643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.566645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.566835 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:12.566945 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:12.567438 (+ 493us) server_negotiation.cc:299] Negotiation successful
0811 05:13:12.567565 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 05:13:12.568786 7811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.554848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42357 (local address 127.7.67.129:45937)
0811 05:13:12.555395 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:12.555445 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:12.557102 (+ 1657us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:12.557674 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:12.557680 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:12.558026 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:12.558486 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:12.558498 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.559656 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.559659 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:12.560748 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:12.560760 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.561481 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.561484 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:12.561625 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.562244 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:12.562261 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:12.563696 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:12.565802 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:12.565815 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:12.565828 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:12.566080 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:12.566391 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:12.566394 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:12.566397 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:12.566537 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:12.566945 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:12.566953 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:12.567199 (+ 246us) client_negotiation.cc:769] Sending connection context
0811 05:13:12.567395 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 05:13:12.567719 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":158,"threads_started":1}
I20260811 05:13:12.570339 7806 heartbeater.cc:347] Connected to a master server at 127.7.67.190:42357
I20260811 05:13:12.570623 7806 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:12.571252 7806 heartbeater.cc:511] Master 127.7.67.190:42357 requested a full tablet report, sending...
I20260811 05:13:12.573058 7509 ts_manager.cc:194] Registered new tserver with Master: 683a8de6d111487ab976faf0c6f7d35d (127.7.67.129:45081)
I20260811 05:13:12.574200 7509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.129@KRBTEST.COM'} at 127.7.67.129:45937
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
I20260811 05:13:12.587409 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:42357
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:12.698391 7814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:12.698622 7814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:12.698663 7814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:12.702870 7814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:12.702922 7814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:12.702979 7814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:13:12.708145 7814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:42357
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:12.709131 7814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:12.710105 7814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:12.719785 7821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:12.719815 7822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:12.719945 7825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:12.719849 7814 server_base.cc:1034] running on GCE node
I20260811 05:13:12.720484 7814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:12.721176 7814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:12.722365 7814 hybrid_clock.cc:648] HybridClock initialized: now 1786425192722350 us; error 29 us; skew 500 ppm
Aug 11 05:13:12 dist-test-slave-04bb krb5kdc[7448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425192, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:12.725190 7814 init.cc:377] Logged in from keytab as kudu/127.7.67.130@KRBTEST.COM (short username kudu)
I20260811 05:13:12.726114 7814 webserver.cc:460] Webserver started at http://127.7.67.130:44585/ using document root <none> and password file <none>
I20260811 05:13:12.726747 7814 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:12.726835 7814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:12.727061 7814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:12.729188 7814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "1fb91654f63548b6bf06358621888bcb"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-04bb"
server_key: "4263fce6eda5c902ce64cc9b917d8816"
server_key_iv: "41728e26d1efea9591aa75745bcfab11"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.729745 7814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "1fb91654f63548b6bf06358621888bcb"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-04bb"
server_key: "4263fce6eda5c902ce64cc9b917d8816"
server_key_iv: "41728e26d1efea9591aa75745bcfab11"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.733456 7814 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:13:12.735975 7835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.737288 7814 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:13:12.737424 7814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "1fb91654f63548b6bf06358621888bcb"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-04bb"
server_key: "4263fce6eda5c902ce64cc9b917d8816"
server_key_iv: "41728e26d1efea9591aa75745bcfab11"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.737534 7814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:12.760062 7814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:12.762943 7814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:12.763144 7814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:12.763717 7814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:12.764603 7814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:12.764668 7814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.764747 7814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:12.764793 7814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.774108 7814 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:33347
I20260811 05:13:12.774196 8016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:33347 every 8 connection(s)
I20260811 05:13:12.775040 7814 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:12.784039 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 7814
I20260811 05:13:12.784171 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
Aug 11 05:13:12 dist-test-slave-04bb krb5kdc[7448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425192, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:12.784498 7438 external_mini_cluster.cc:1428] Setting key 6849d6ccc78fe328e44ee6b1bb57a23c
I20260811 05:13:12.788955 7571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.776916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:58011 (local address 127.7.67.190:42357)
0811 05:13:12.777048 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:12.777052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:12.777847 (+ 795us) server_negotiation.cc:407] Connection header received
0811 05:13:12.779113 (+ 1266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:12.779116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:12.779166 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:12.779245 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:12.780354 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.780930 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.782042 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.782253 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.782333 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.785090 (+ 2757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:12.785098 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:12.785103 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:12.785136 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:12.786599 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.787225 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.787227 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.787228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.787303 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:12.787688 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:12.787690 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:12.787691 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:12.787858 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:12.787944 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:12.788606 (+ 662us) server_negotiation.cc:299] Negotiation successful
0811 05:13:12.788741 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 05:13:12.789659 8019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.777228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42357 (local address 127.7.67.130:58011)
0811 05:13:12.777691 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:12.777732 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:12.778916 (+ 1184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:12.779421 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:12.779427 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:12.779764 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:12.780216 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:12.780229 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.781038 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.781041 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:12.781932 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:12.781941 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:12.782407 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:12.782411 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:12.782553 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:12.783174 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:12.783189 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:12.784902 (+ 1713us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:12.786727 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:12.786736 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:12.786751 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:12.787122 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:12.787394 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:12.787400 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:12.787405 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:12.787592 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:12.787975 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:12.787984 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:12.788304 (+ 320us) client_negotiation.cc:769] Sending connection context
0811 05:13:12.788492 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 05:13:12.788809 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":112,"threads_started":1}
I20260811 05:13:12.790925 8017 heartbeater.cc:347] Connected to a master server at 127.7.67.190:42357
I20260811 05:13:12.791162 8017 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:12.791723 8017 heartbeater.cc:511] Master 127.7.67.190:42357 requested a full tablet report, sending...
I20260811 05:13:12.792932 7509 ts_manager.cc:194] Registered new tserver with Master: 1fb91654f63548b6bf06358621888bcb (127.7.67.130:33347)
I20260811 05:13:12.793810 7509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.130@KRBTEST.COM'} at 127.7.67.130:58011
WARNING: no policy specified for kudu/127.7.67.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
I20260811 05:13:12.815328 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:42357
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:34615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:12.930231 8022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:12.930502 8022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:12.930577 8022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:12.934527 8022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:12.934613 8022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:12.934706 8022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:13:12.939931 8022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:34615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:42357
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:12.941088 8022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:12.942178 8022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:12.952083 8034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:12.952092 8032 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:12.952088 8031 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:12.952263 8022 server_base.cc:1034] running on GCE node
I20260811 05:13:12.952884 8022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:12.953555 8022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:12.954744 8022 hybrid_clock.cc:648] HybridClock initialized: now 1786425192954721 us; error 31 us; skew 500 ppm
Aug 11 05:13:12 dist-test-slave-04bb krb5kdc[7448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425192, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:12.957683 8022 init.cc:377] Logged in from keytab as kudu/127.7.67.131@KRBTEST.COM (short username kudu)
I20260811 05:13:12.958477 8022 webserver.cc:460] Webserver started at http://127.7.67.131:41693/ using document root <none> and password file <none>
I20260811 05:13:12.958993 8022 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:12.959052 8022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:12.959213 8022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:12.961380 8022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "feb7412cc24347bfbdd8e79910553433"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-04bb"
server_key: "775deab84de9fbf8288f8c429045b3ec"
server_key_iv: "733500b0307cc19a5838f46ed86fe30e"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.961899 8022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "feb7412cc24347bfbdd8e79910553433"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-04bb"
server_key: "775deab84de9fbf8288f8c429045b3ec"
server_key_iv: "733500b0307cc19a5838f46ed86fe30e"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.965595 8022 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 05:13:12.967897 8046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.969182 8022 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:13:12.969300 8022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "feb7412cc24347bfbdd8e79910553433"
format_stamp: "Formatted at 2026-08-11 05:13:12 on dist-test-slave-04bb"
server_key: "775deab84de9fbf8288f8c429045b3ec"
server_key_iv: "733500b0307cc19a5838f46ed86fe30e"
server_key_version: "encryptionkey@0"
I20260811 05:13:12.969384 8022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:12.982331 8022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:12.985482 8022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:12.985628 8022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:12.986168 8022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:12.986923 8022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:12.986971 8022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.987008 8022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:12.987023 8022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:12.996237 8022 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:46221
I20260811 05:13:12.996301 8227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:46221 every 8 connection(s)
I20260811 05:13:12.997151 8022 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:13:13.001977 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 8022
I20260811 05:13:13.002107 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:13:13.002430 7438 external_mini_cluster.cc:1428] Setting key 5d77c09267c3d1d202a5a668ba6f99c6
Aug 11 05:13:13 dist-test-slave-04bb krb5kdc[7448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425192, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:13.011531 7571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.999002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:59029 (local address 127.7.67.190:42357)
0811 05:13:12.999178 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:12.999184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.000043 (+ 859us) server_negotiation.cc:407] Connection header received
0811 05:13:13.001539 (+ 1496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.001542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.001590 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.001687 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.003334 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.003994 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.005554 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.005782 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.005873 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.008325 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.008328 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.008330 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.008355 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.009789 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.010242 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.010244 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.010245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.010320 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.010605 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.010607 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.010609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.010787 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.010865 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.011299 (+ 434us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.011396 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 05:13:13.012337 8231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:12.999432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42357 (local address 127.7.67.131:59029)
0811 05:13:12.999904 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:12.999947 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:13.001318 (+ 1371us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:13.001922 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:13.001932 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:13.002524 (+ 592us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:13.003160 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.003176 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.004124 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.004127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.005440 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:13.005450 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.005924 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.005927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:13.006065 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.006689 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:13.006706 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:13.008150 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:13.009894 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.009899 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.009910 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.010154 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.010398 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:13.010400 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:13.010402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:13.010527 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:13.010875 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:13.010880 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:13.011104 (+ 224us) client_negotiation.cc:769] Sending connection context
0811 05:13:13.011274 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 05:13:13.011528 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":126,"threads_started":1}
I20260811 05:13:13.013536 8228 heartbeater.cc:347] Connected to a master server at 127.7.67.190:42357
I20260811 05:13:13.013799 8228 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:13.014317 8228 heartbeater.cc:511] Master 127.7.67.190:42357 requested a full tablet report, sending...
I20260811 05:13:13.015370 7509 ts_manager.cc:194] Registered new tserver with Master: feb7412cc24347bfbdd8e79910553433 (127.7.67.131:46221)
I20260811 05:13:13.016052 7509 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.131@KRBTEST.COM'} at 127.7.67.131:59029
I20260811 05:13:13.017313 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 05:13:13 dist-test-slave-04bb krb5kdc[7448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425193, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 05:13:13 dist-test-slave-04bb krb5kdc[7448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425193, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.67.129@KRBTEST.COM
I20260811 05:13:13.045105 8238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.035600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38048 (local address 127.7.67.129:45081)
0811 05:13:13.035924 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.035939 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.035963 (+ 24us) server_negotiation.cc:407] Connection header received
0811 05:13:13.036099 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.036104 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.036290 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.036411 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.037139 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.037914 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.038694 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.039052 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.039135 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.041672 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.041681 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.041694 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.041716 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.043216 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.043673 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.043678 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.043679 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.043769 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.044061 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.044065 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.044066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.044310 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.044563 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.044779 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.044934 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":228,"thread_start_us":127,"threads_started":1}
W20260811 05:13:13.046304 7672 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38048
Aug 11 05:13:13 dist-test-slave-04bb krb5kdc[7448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 05:13:13 dist-test-slave-04bb krb5kdc[7448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.129@KRBTEST.COM
I20260811 05:13:13.074414 8238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:13.065541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38050 (local address 127.7.67.129:45081)
0811 05:13:13.065733 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:13.065738 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:13.065864 (+ 126us) server_negotiation.cc:407] Connection header received
0811 05:13:13.066066 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:13.066069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:13.066111 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:13.066198 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:13.066947 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.067453 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.068311 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:13.068596 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:13.068706 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:13.071079 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:13.071083 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:13.071090 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:13.071118 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:13.072434 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.073054 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.073058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.073065 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.073165 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:13.073475 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:13.073491 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:13.073493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:13.073678 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:13.073822 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:13.074078 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 05:13:13.074230 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 05:13:13.075598 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 7606
I20260811 05:13:13.089852 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 7814
I20260811 05:13:13.100617 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 8022
I20260811 05:13:13.112964 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 7468
2026-08-11T05:13:13Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2447 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:13:13 dist-test-slave-04bb krb5kdc[8246](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:13:13 dist-test-slave-04bb krb5kdc[8246](info): set up 2 sockets
Aug 11 05:13:13 dist-test-slave-04bb krb5kdc[8246](info): commencing operation
krb5kdc: starting...
W20260811 05:13:14.377700 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:13:14 dist-test-slave-04bb krb5kdc[8246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:13:14.505767 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:41845
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:37575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:41845
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:14.618371 8263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:14.618621 8263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:14.618669 8263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:14.622603 8263 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:14.622655 8263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:14.622675 8263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:14.622692 8263 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:14.622749 8263 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:14.627823 8263 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:37575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:41845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:41845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:14.628938 8263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:14.629973 8263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:14.639026 8271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:14.639037 8268 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:14.639146 8263 server_base.cc:1034] running on GCE node
W20260811 05:13:14.639045 8274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:14.639959 8263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:14.641148 8263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:14.642341 8263 hybrid_clock.cc:648] HybridClock initialized: now 1786425194642308 us; error 46 us; skew 500 ppm
Aug 11 05:13:14 dist-test-slave-04bb krb5kdc[8246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425194, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:14.649672 8263 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:13:14.650629 8263 webserver.cc:460] Webserver started at http://127.7.67.190:40629/ using document root <none> and password file <none>
I20260811 05:13:14.651207 8263 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:14.651306 8263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:14.651549 8263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:14.653746 8263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "0c84fdb5d83e41609247c2f88b97ef7d"
format_stamp: "Formatted at 2026-08-11 05:13:14 on dist-test-slave-04bb"
server_key: "905d8c95e8d1ea77f14bb7a0b2fb5a89"
server_key_iv: "820a11a90557103485499ec2e03301ae"
server_key_version: "encryptionkey@0"
I20260811 05:13:14.654325 8263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "0c84fdb5d83e41609247c2f88b97ef7d"
format_stamp: "Formatted at 2026-08-11 05:13:14 on dist-test-slave-04bb"
server_key: "905d8c95e8d1ea77f14bb7a0b2fb5a89"
server_key_iv: "820a11a90557103485499ec2e03301ae"
server_key_version: "encryptionkey@0"
I20260811 05:13:14.658059 8263 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:13:14.660485 8281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:14.661792 8263 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:13:14.661929 8263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "0c84fdb5d83e41609247c2f88b97ef7d"
format_stamp: "Formatted at 2026-08-11 05:13:14 on dist-test-slave-04bb"
server_key: "905d8c95e8d1ea77f14bb7a0b2fb5a89"
server_key_iv: "820a11a90557103485499ec2e03301ae"
server_key_version: "encryptionkey@0"
I20260811 05:13:14.662036 8263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:14.676687 8263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:14.679405 8263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:14.679597 8263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:14.686513 8263 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:41845
I20260811 05:13:14.686586 8372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:41845 every 8 connection(s)
I20260811 05:13:14.687461 8263 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:14.690320 8373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:14.692260 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 8263
I20260811 05:13:14.692376 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:13:14.692703 7438 external_mini_cluster.cc:1428] Setting key ba77a6bfc2fbc05ddb619d8a98d170a3
I20260811 05:13:14.697125 8373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c84fdb5d83e41609247c2f88b97ef7d: Bootstrap starting.
I20260811 05:13:14.699803 8373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c84fdb5d83e41609247c2f88b97ef7d: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:14.700788 8373 log.cc:824] T 00000000000000000000000000000000 P 0c84fdb5d83e41609247c2f88b97ef7d: Log is configured to *not* fsync() on all Append() calls
Aug 11 05:13:14 dist-test-slave-04bb krb5kdc[8246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:14.702832 8373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c84fdb5d83e41609247c2f88b97ef7d: No bootstrap required, opened a new log
I20260811 05:13:14.707811 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:14.694101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54768 (local address 127.7.67.190:41845)
0811 05:13:14.694509 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:14.694526 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:14.694562 (+ 36us) server_negotiation.cc:407] Connection header received
0811 05:13:14.695177 (+ 615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:14.695194 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:14.695512 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:14.695781 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:14.696669 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:14.697773 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:14.698959 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:14.699481 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:14.699604 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:14.702562 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:14.702567 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:14.702580 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:14.702606 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:14.704610 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:14.705063 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:14.705069 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:14.705075 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:14.705234 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:14.705552 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:14.705558 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:14.705560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:14.706079 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:14.706275 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:14.706544 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 05:13:14.706854 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":130,"threads_started":1}
I20260811 05:13:14.707687 8373 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c84fdb5d83e41609247c2f88b97ef7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c84fdb5d83e41609247c2f88b97ef7d" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 41845 } }
I20260811 05:13:14.707959 8373 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c84fdb5d83e41609247c2f88b97ef7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:14.708011 8373 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c84fdb5d83e41609247c2f88b97ef7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c84fdb5d83e41609247c2f88b97ef7d, State: Initialized, Role: FOLLOWER
I20260811 05:13:14.708438 8373 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c84fdb5d83e41609247c2f88b97ef7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c84fdb5d83e41609247c2f88b97ef7d" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 41845 } }
I20260811 05:13:14.708619 8373 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c84fdb5d83e41609247c2f88b97ef7d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:14.708693 8373 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c84fdb5d83e41609247c2f88b97ef7d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:14.708791 8373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c84fdb5d83e41609247c2f88b97ef7d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:14.709913 8373 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c84fdb5d83e41609247c2f88b97ef7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c84fdb5d83e41609247c2f88b97ef7d" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 41845 } }
I20260811 05:13:14.710278 8373 leader_election.cc:302] T 00000000000000000000000000000000 P 0c84fdb5d83e41609247c2f88b97ef7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c84fdb5d83e41609247c2f88b97ef7d; no voters:
I20260811 05:13:14.710620 8373 leader_election.cc:288] T 00000000000000000000000000000000 P 0c84fdb5d83e41609247c2f88b97ef7d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:14.710690 8380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c84fdb5d83e41609247c2f88b97ef7d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:14.711117 8380 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c84fdb5d83e41609247c2f88b97ef7d [term 1 LEADER]: Becoming Leader. State: Replica: 0c84fdb5d83e41609247c2f88b97ef7d, State: Running, Role: LEADER
I20260811 05:13:14.711421 8380 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c84fdb5d83e41609247c2f88b97ef7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c84fdb5d83e41609247c2f88b97ef7d" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 41845 } }
I20260811 05:13:14.711927 8373 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c84fdb5d83e41609247c2f88b97ef7d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:14.713268 8383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c84fdb5d83e41609247c2f88b97ef7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c84fdb5d83e41609247c2f88b97ef7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c84fdb5d83e41609247c2f88b97ef7d" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 41845 } } }
I20260811 05:13:14.713424 8383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c84fdb5d83e41609247c2f88b97ef7d [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:14.713862 8382 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c84fdb5d83e41609247c2f88b97ef7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c84fdb5d83e41609247c2f88b97ef7d. Latest consensus state: current_term: 1 leader_uuid: "0c84fdb5d83e41609247c2f88b97ef7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c84fdb5d83e41609247c2f88b97ef7d" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 41845 } } }
I20260811 05:13:14.714012 8394 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:14.714540 8382 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c84fdb5d83e41609247c2f88b97ef7d [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:14.717162 8394 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:14.723179 8394 catalog_manager.cc:1347] Generated new cluster ID: 31778522a6ab4fbe8ea34b364128c948
I20260811 05:13:14.723251 8394 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:14.750783 8394 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:14.751802 8394 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:14.759804 8394 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c84fdb5d83e41609247c2f88b97ef7d: Generated new TSK 0
I20260811 05:13:14.760514 8394 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
I20260811 05:13:14.795737 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:41845
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:37575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:14.909746 8409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:14.909999 8409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:14.910043 8409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:14.914505 8409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:14.914559 8409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:14.914618 8409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:13:14.919785 8409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:37575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:41845
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:14.920895 8409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:14.921931 8409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:14.932174 8416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:14.932322 8420 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:14.932128 8417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:14.932618 8409 server_base.cc:1034] running on GCE node
I20260811 05:13:14.932982 8409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:14.933604 8409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:14.934794 8409 hybrid_clock.cc:648] HybridClock initialized: now 1786425194934779 us; error 29 us; skew 500 ppm
Aug 11 05:13:14 dist-test-slave-04bb krb5kdc[8246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425194, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:14.937661 8409 init.cc:377] Logged in from keytab as kudu/127.7.67.129@KRBTEST.COM (short username kudu)
I20260811 05:13:14.938535 8409 webserver.cc:460] Webserver started at http://127.7.67.129:42291/ using document root <none> and password file <none>
I20260811 05:13:14.939112 8409 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:14.939206 8409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:14.939464 8409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:14.941695 8409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "4e45a2d12600427c9add4b3d9919293d"
format_stamp: "Formatted at 2026-08-11 05:13:14 on dist-test-slave-04bb"
server_key: "05acc4d26814a3dc28709ee7201ea49f"
server_key_iv: "a1caac5784b132d65507eeebc055d4ff"
server_key_version: "encryptionkey@0"
I20260811 05:13:14.942271 8409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "4e45a2d12600427c9add4b3d9919293d"
format_stamp: "Formatted at 2026-08-11 05:13:14 on dist-test-slave-04bb"
server_key: "05acc4d26814a3dc28709ee7201ea49f"
server_key_iv: "a1caac5784b132d65507eeebc055d4ff"
server_key_version: "encryptionkey@0"
I20260811 05:13:14.946070 8409 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 05:13:14.948518 8428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:14.949765 8409 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:13:14.949905 8409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "4e45a2d12600427c9add4b3d9919293d"
format_stamp: "Formatted at 2026-08-11 05:13:14 on dist-test-slave-04bb"
server_key: "05acc4d26814a3dc28709ee7201ea49f"
server_key_iv: "a1caac5784b132d65507eeebc055d4ff"
server_key_version: "encryptionkey@0"
I20260811 05:13:14.950021 8409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:14.963423 8409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:14.966531 8409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:14.966733 8409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:14.967352 8409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:14.968286 8409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:14.968353 8409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:14.968400 8409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:14.968429 8409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:14.977833 8409 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:41963
I20260811 05:13:14.977866 8604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:41963 every 8 connection(s)
I20260811 05:13:14.978727 8409 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:14.982342 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 8409
I20260811 05:13:14.982434 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:13:14.982686 7438 external_mini_cluster.cc:1428] Setting key 2f86eef8423e89f6025ab4cd0a348eb5
Aug 11 05:13:14 dist-test-slave-04bb krb5kdc[8246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425194, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:14.994645 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:14.980641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:58281 (local address 127.7.67.190:41845)
0811 05:13:14.980796 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:14.980800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:14.981610 (+ 810us) server_negotiation.cc:407] Connection header received
0811 05:13:14.983197 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:14.983201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:14.983278 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:14.983397 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:14.984710 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:14.985296 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:14.987042 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:14.987277 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:14.987369 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:14.990259 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:14.990266 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:14.990269 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:14.990300 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:14.992164 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:14.992801 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:14.992808 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:14.992813 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:14.992920 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:14.993278 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:14.993284 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:14.993286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:14.993574 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:14.993693 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:14.994217 (+ 524us) server_negotiation.cc:299] Negotiation successful
0811 05:13:14.994393 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
WARNING: no policy specified for kudu/127.7.67.130@KRBTEST.COM; defaulting to no policy
I20260811 05:13:14.995270 8608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:14.980977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:41845 (local address 127.7.67.129:58281)
0811 05:13:14.981442 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:14.981488 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:14.982968 (+ 1480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:14.983591 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:14.983598 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:14.983984 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:14.984503 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:14.984541 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:14.985439 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:14.985443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:14.986896 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:14.986913 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:14.987456 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:14.987462 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:14.987672 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:14.988280 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:14.988309 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:14.990054 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:14.992320 (+ 2266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:14.992325 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:14.992337 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:14.992676 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:14.993024 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:14.993026 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:14.993028 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:14.993160 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:14.993693 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:14.993698 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:14.993929 (+ 231us) client_negotiation.cc:769] Sending connection context
0811 05:13:14.994086 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 05:13:14.994349 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":110,"threads_started":1}
I20260811 05:13:14.996600 8605 heartbeater.cc:347] Connected to a master server at 127.7.67.190:41845
I20260811 05:13:14.996877 8605 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:14.997444 8605 heartbeater.cc:511] Master 127.7.67.190:41845 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.130@KRBTEST.COM" created.
I20260811 05:13:14.999068 8312 ts_manager.cc:194] Registered new tserver with Master: 4e45a2d12600427c9add4b3d9919293d (127.7.67.129:41963)
I20260811 05:13:15.000207 8312 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.129@KRBTEST.COM'} at 127.7.67.129:58281
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
I20260811 05:13:15.015390 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:41845
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:37575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:15.140760 8613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:15.140996 8613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:15.141039 8613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:15.145068 8613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:15.145119 8613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:15.145228 8613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:13:15.150363 8613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:37575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:41845
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:15.151369 8613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:15.152412 8613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:15.162075 8622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:15.162098 8625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:15.162104 8621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:15.162170 8613 server_base.cc:1034] running on GCE node
I20260811 05:13:15.162685 8613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:15.163259 8613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:15.164417 8613 hybrid_clock.cc:648] HybridClock initialized: now 1786425195164387 us; error 41 us; skew 500 ppm
Aug 11 05:13:15 dist-test-slave-04bb krb5kdc[8246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425195, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:15.167413 8613 init.cc:377] Logged in from keytab as kudu/127.7.67.130@KRBTEST.COM (short username kudu)
I20260811 05:13:15.168393 8613 webserver.cc:460] Webserver started at http://127.7.67.130:46705/ using document root <none> and password file <none>
I20260811 05:13:15.168979 8613 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:15.169044 8613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:15.169265 8613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:15.171315 8613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "c84db6206adf42b4b0983f845ae33d25"
format_stamp: "Formatted at 2026-08-11 05:13:15 on dist-test-slave-04bb"
server_key: "c5ce8e9b06c73a950c3dd18bdcfbe4c6"
server_key_iv: "260dbba7217daf7e83ca2ae982aeff27"
server_key_version: "encryptionkey@0"
I20260811 05:13:15.171887 8613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "c84db6206adf42b4b0983f845ae33d25"
format_stamp: "Formatted at 2026-08-11 05:13:15 on dist-test-slave-04bb"
server_key: "c5ce8e9b06c73a950c3dd18bdcfbe4c6"
server_key_iv: "260dbba7217daf7e83ca2ae982aeff27"
server_key_version: "encryptionkey@0"
I20260811 05:13:15.175678 8613 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 05:13:15.178149 8633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:15.179390 8613 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:15.179536 8613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "c84db6206adf42b4b0983f845ae33d25"
format_stamp: "Formatted at 2026-08-11 05:13:15 on dist-test-slave-04bb"
server_key: "c5ce8e9b06c73a950c3dd18bdcfbe4c6"
server_key_iv: "260dbba7217daf7e83ca2ae982aeff27"
server_key_version: "encryptionkey@0"
I20260811 05:13:15.179648 8613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:15.194768 8613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:15.197789 8613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:15.198030 8613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:15.198674 8613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:15.199577 8613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:15.199658 8613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:15.199736 8613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:15.199785 8613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:15.210492 8613 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:44017
I20260811 05:13:15.210537 8822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:44017 every 8 connection(s)
I20260811 05:13:15.211443 8613 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:15.217000 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 8613
I20260811 05:13:15.217129 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:13:15.217424 7438 external_mini_cluster.cc:1428] Setting key efe4a4b12ced10bf2617fba1f6d1ceec
Aug 11 05:13:15 dist-test-slave-04bb krb5kdc[8246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425195, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:15.228436 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.213646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:33535 (local address 127.7.67.190:41845)
0811 05:13:15.213795 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:15.213799 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:15.214761 (+ 962us) server_negotiation.cc:407] Connection header received
0811 05:13:15.216182 (+ 1421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:15.216185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:15.216242 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:15.216319 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:15.218003 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.218766 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.220652 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.220884 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.220978 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:15.223859 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:15.223866 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:15.223872 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:15.223908 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:15.226057 (+ 2149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:15.226675 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:15.226681 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:15.226686 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:15.226793 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:15.227162 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:15.227167 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:15.227168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:15.227424 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:15.227542 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:15.228084 (+ 542us) server_negotiation.cc:299] Negotiation successful
0811 05:13:15.228241 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
WARNING: no policy specified for kudu/127.7.67.131@KRBTEST.COM; defaulting to no policy
I20260811 05:13:15.229394 8830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.214013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:41845 (local address 127.7.67.130:33535)
0811 05:13:15.214623 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:15.214668 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:15.215963 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:15.216464 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:15.216471 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:15.217048 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:15.217819 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:15.217843 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.218915 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.218918 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:15.220482 (+ 1564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:15.220498 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.221055 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.221062 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:15.221336 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:15.222094 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:15.222112 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:15.223671 (+ 1559us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:15.226242 (+ 2571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:15.226247 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:15.226261 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:15.226546 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:15.226906 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:15.226908 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:15.226910 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:15.227052 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:15.227551 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:15.227556 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:15.227808 (+ 252us) client_negotiation.cc:769] Sending connection context
0811 05:13:15.227985 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 05:13:15.228287 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":170,"threads_started":1}
I20260811 05:13:15.230643 8824 heartbeater.cc:347] Connected to a master server at 127.7.67.190:41845
I20260811 05:13:15.230969 8824 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.131@KRBTEST.COM" created.
I20260811 05:13:15.231529 8824 heartbeater.cc:511] Master 127.7.67.190:41845 requested a full tablet report, sending...
I20260811 05:13:15.232823 8312 ts_manager.cc:194] Registered new tserver with Master: c84db6206adf42b4b0983f845ae33d25 (127.7.67.130:44017)
I20260811 05:13:15.233457 8312 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.130@KRBTEST.COM'} at 127.7.67.130:33535
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
I20260811 05:13:15.247999 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:41845
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:37575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:15.365290 8834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:15.365532 8834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:15.365607 8834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:15.369684 8834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:15.369742 8834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:15.369843 8834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:13:15.374950 8834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:37575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:41845
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:15.375946 8834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:15.377049 8834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:15.387169 8839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:15.387178 8843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:15.387245 8840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:15.387503 8834 server_base.cc:1034] running on GCE node
I20260811 05:13:15.387928 8834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:15.388690 8834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:15.389894 8834 hybrid_clock.cc:648] HybridClock initialized: now 1786425195389874 us; error 31 us; skew 500 ppm
Aug 11 05:13:15 dist-test-slave-04bb krb5kdc[8246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425195, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:15.392741 8834 init.cc:377] Logged in from keytab as kudu/127.7.67.131@KRBTEST.COM (short username kudu)
I20260811 05:13:15.393734 8834 webserver.cc:460] Webserver started at http://127.7.67.131:42965/ using document root <none> and password file <none>
I20260811 05:13:15.394342 8834 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:15.394443 8834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:15.394685 8834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:15.396906 8834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "f9ff216553e04ca8b5ae099e835a9a94"
format_stamp: "Formatted at 2026-08-11 05:13:15 on dist-test-slave-04bb"
server_key: "ede3190f185e28b8a0347c6911116016"
server_key_iv: "85c6ab71a3ed7ba73f1858175df677ee"
server_key_version: "encryptionkey@0"
I20260811 05:13:15.397539 8834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "f9ff216553e04ca8b5ae099e835a9a94"
format_stamp: "Formatted at 2026-08-11 05:13:15 on dist-test-slave-04bb"
server_key: "ede3190f185e28b8a0347c6911116016"
server_key_iv: "85c6ab71a3ed7ba73f1858175df677ee"
server_key_version: "encryptionkey@0"
I20260811 05:13:15.401543 8834 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 05:13:15.404187 8854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:15.405427 8834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:13:15.405581 8834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "f9ff216553e04ca8b5ae099e835a9a94"
format_stamp: "Formatted at 2026-08-11 05:13:15 on dist-test-slave-04bb"
server_key: "ede3190f185e28b8a0347c6911116016"
server_key_iv: "85c6ab71a3ed7ba73f1858175df677ee"
server_key_version: "encryptionkey@0"
I20260811 05:13:15.405691 8834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:15.421108 8834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:15.424193 8834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:15.424401 8834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:15.425056 8834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:15.425905 8834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:15.425980 8834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:15.426054 8834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:15.426106 8834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:15.435379 8834 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:43357
I20260811 05:13:15.435400 9035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:43357 every 8 connection(s)
I20260811 05:13:15.436334 8834 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:13:15.444494 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 8834
I20260811 05:13:15.444681 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:13:15.444967 7438 external_mini_cluster.cc:1428] Setting key c7c93325327402928a1e56433b3b4a3c
Aug 11 05:13:15 dist-test-slave-04bb krb5kdc[8246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425195, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:15.450577 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.438306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:56103 (local address 127.7.67.190:41845)
0811 05:13:15.438470 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:15.438474 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:15.439313 (+ 839us) server_negotiation.cc:407] Connection header received
0811 05:13:15.440765 (+ 1452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:15.440769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:15.440831 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:15.440941 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:15.442131 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.442676 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.443815 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.444038 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.444153 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:15.447074 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:15.447078 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:15.447081 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:15.447110 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:15.448724 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:15.449259 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:15.449262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:15.449263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:15.449346 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:15.449649 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:15.449652 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:15.449653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:15.449829 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:15.449907 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:15.450365 (+ 458us) server_negotiation.cc:299] Negotiation successful
0811 05:13:15.450461 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 05:13:15.451377 9038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.438654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:41845 (local address 127.7.67.131:56103)
0811 05:13:15.439135 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:15.439186 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:15.440504 (+ 1318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:15.441126 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:15.441134 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:15.441498 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:15.441982 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:15.441996 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.442806 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.442811 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:15.443686 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:15.443695 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.444193 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.444198 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:15.444392 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:15.445100 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:15.445125 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:15.446903 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:15.448830 (+ 1927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:15.448835 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:15.448847 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:15.449133 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:15.449432 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:15.449435 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:15.449436 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:15.449566 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:15.449919 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:15.449923 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:15.450148 (+ 225us) client_negotiation.cc:769] Sending connection context
0811 05:13:15.450313 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 05:13:15.450559 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":113,"threads_started":1}
I20260811 05:13:15.452590 9036 heartbeater.cc:347] Connected to a master server at 127.7.67.190:41845
I20260811 05:13:15.452848 9036 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:15.453342 9036 heartbeater.cc:511] Master 127.7.67.190:41845 requested a full tablet report, sending...
I20260811 05:13:15.454563 8312 ts_manager.cc:194] Registered new tserver with Master: f9ff216553e04ca8b5ae099e835a9a94 (127.7.67.131:43357)
I20260811 05:13:15.455178 8312 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.131@KRBTEST.COM'} at 127.7.67.131:56103
I20260811 05:13:15.458947 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:13:15.469383 8377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.462328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42368 (local address 127.7.67.190:41845)
0811 05:13:15.462450 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:15.462455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:15.462626 (+ 171us) server_negotiation.cc:407] Connection header received
0811 05:13:15.462724 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:15.462726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:15.462768 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:15.462830 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:15.463491 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.463976 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.464869 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.465071 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.465204 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:15.466568 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:15.466571 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:15.466573 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:15.466592 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:15.467804 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:15.468186 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:15.468188 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:15.468190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:15.468263 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:15.468595 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:15.468597 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:15.468598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:15.468805 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:15.468911 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:15.469115 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 05:13:15.469203 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 05:13:15.473954 8312 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42368:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:13:15.476963 8312 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:15.491138 9050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.485397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:44017 (local address 127.0.0.1:35268)
0811 05:13:15.485877 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:15.485926 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:15.486096 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:15.486969 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:15.486976 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:15.487004 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:15.487148 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:15.487156 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.488433 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.488440 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:15.489456 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:15.489481 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.490590 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.490597 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:15.490710 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:15.490756 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 05:13:15.490870 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 05:13:15.490983 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":59,"threads_started":1}
I20260811 05:13:15.491168 9051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.485527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35268 (local address 127.7.67.130:44017)
0811 05:13:15.485883 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:15.485893 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:15.486015 (+ 122us) server_negotiation.cc:407] Connection header received
0811 05:13:15.486334 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:15.486341 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:15.486563 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:15.486735 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:15.487289 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.488306 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.489604 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.490459 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.490583 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:15.490744 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:15.490920 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 05:13:15.491016 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":162,"threads_started":1}
I20260811 05:13:15.491950 9052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.486468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:43357 (local address 127.0.0.1:36350)
0811 05:13:15.486861 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:15.486874 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:15.486945 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:15.487773 (+ 828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:15.487778 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:15.487797 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:15.487902 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:15.487908 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.489277 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.489281 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:15.490541 (+ 1260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:15.490557 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.491671 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.491679 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:15.491754 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:15.491775 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 05:13:15.491815 (+ 40us) client_negotiation.cc:240] Negotiation successful
0811 05:13:15.491860 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"mutex_wait_us":96,"thread_start_us":95,"threads_started":1}
I20260811 05:13:15.492216 9054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.486135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:41963 (local address 127.0.0.1:40896)
0811 05:13:15.486705 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:15.486719 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:15.486837 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:15.487566 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:15.487569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:15.487585 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:15.487679 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:15.487689 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.489246 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.489250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:15.490530 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:15.490543 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.491840 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.491844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:15.491930 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:15.491952 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 05:13:15.492023 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 05:13:15.492083 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"mutex_wait_us":42,"thread_start_us":118,"threads_started":1}
I20260811 05:13:15.492735 9055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.486896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36350 (local address 127.7.67.131:43357)
0811 05:13:15.487329 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:15.487344 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:15.487367 (+ 23us) server_negotiation.cc:407] Connection header received
0811 05:13:15.487439 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:15.487446 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:15.487619 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:15.487776 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:15.488021 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.489137 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.490694 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.491525 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.492245 (+ 720us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:15.492419 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:15.492475 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 05:13:15.492608 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":169,"threads_started":1}
I20260811 05:13:15.492727 9053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.486359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40896 (local address 127.7.67.129:41963)
0811 05:13:15.486725 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:15.486741 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:15.486937 (+ 196us) server_negotiation.cc:407] Connection header received
0811 05:13:15.487044 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:15.487049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:15.487258 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:15.487409 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:15.487829 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.489080 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.490661 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.491511 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.491619 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:15.491774 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:15.492447 (+ 673us) server_negotiation.cc:299] Negotiation successful
0811 05:13:15.492581 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":74,"threads_started":1}
I20260811 05:13:15.494529 8719 tablet_service.cc:1467] Processing CreateTablet for tablet fb0794d2a14b4971bea153dd3d582569 (DEFAULT_TABLE table=test-table [id=061b288a206f4e24af007b09abf10e43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:15.495705 8719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb0794d2a14b4971bea153dd3d582569. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:15.495491 8928 tablet_service.cc:1467] Processing CreateTablet for tablet fb0794d2a14b4971bea153dd3d582569 (DEFAULT_TABLE table=test-table [id=061b288a206f4e24af007b09abf10e43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:15.496503 8928 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb0794d2a14b4971bea153dd3d582569. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:15.497578 8497 tablet_service.cc:1467] Processing CreateTablet for tablet fb0794d2a14b4971bea153dd3d582569 (DEFAULT_TABLE table=test-table [id=061b288a206f4e24af007b09abf10e43]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:15.498498 8497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb0794d2a14b4971bea153dd3d582569. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:15.503408 9056 tablet_bootstrap.cc:492] T fb0794d2a14b4971bea153dd3d582569 P c84db6206adf42b4b0983f845ae33d25: Bootstrap starting.
I20260811 05:13:15.503859 9057 tablet_bootstrap.cc:492] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94: Bootstrap starting.
I20260811 05:13:15.504091 9059 tablet_bootstrap.cc:492] T fb0794d2a14b4971bea153dd3d582569 P 4e45a2d12600427c9add4b3d9919293d: Bootstrap starting.
I20260811 05:13:15.505985 9057 tablet_bootstrap.cc:654] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:15.506012 9059 tablet_bootstrap.cc:654] T fb0794d2a14b4971bea153dd3d582569 P 4e45a2d12600427c9add4b3d9919293d: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:15.506731 9059 log.cc:824] T fb0794d2a14b4971bea153dd3d582569 P 4e45a2d12600427c9add4b3d9919293d: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:15.507299 9057 log.cc:824] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:15.508809 9059 tablet_bootstrap.cc:492] T fb0794d2a14b4971bea153dd3d582569 P 4e45a2d12600427c9add4b3d9919293d: No bootstrap required, opened a new log
I20260811 05:13:15.509073 9059 ts_tablet_manager.cc:1397] T fb0794d2a14b4971bea153dd3d582569 P 4e45a2d12600427c9add4b3d9919293d: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 05:13:15.509336 9057 tablet_bootstrap.cc:492] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94: No bootstrap required, opened a new log
I20260811 05:13:15.509413 9056 tablet_bootstrap.cc:654] T fb0794d2a14b4971bea153dd3d582569 P c84db6206adf42b4b0983f845ae33d25: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:15.509526 9057 ts_tablet_manager.cc:1397] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 05:13:15.510234 9056 log.cc:824] T fb0794d2a14b4971bea153dd3d582569 P c84db6206adf42b4b0983f845ae33d25: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:15.512598 9056 tablet_bootstrap.cc:492] T fb0794d2a14b4971bea153dd3d582569 P c84db6206adf42b4b0983f845ae33d25: No bootstrap required, opened a new log
I20260811 05:13:15.512825 9056 ts_tablet_manager.cc:1397] T fb0794d2a14b4971bea153dd3d582569 P c84db6206adf42b4b0983f845ae33d25: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.000s
I20260811 05:13:15.513511 9057 raft_consensus.cc:348] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84db6206adf42b4b0983f845ae33d25" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44017 } } peers { permanent_uuid: "4e45a2d12600427c9add4b3d9919293d" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 41963 } } peers { permanent_uuid: "f9ff216553e04ca8b5ae099e835a9a94" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43357 } }
I20260811 05:13:15.513511 9059 raft_consensus.cc:348] T fb0794d2a14b4971bea153dd3d582569 P 4e45a2d12600427c9add4b3d9919293d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84db6206adf42b4b0983f845ae33d25" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44017 } } peers { permanent_uuid: "4e45a2d12600427c9add4b3d9919293d" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 41963 } } peers { permanent_uuid: "f9ff216553e04ca8b5ae099e835a9a94" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43357 } }
I20260811 05:13:15.513788 9059 raft_consensus.cc:374] T fb0794d2a14b4971bea153dd3d582569 P 4e45a2d12600427c9add4b3d9919293d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:15.513788 9057 raft_consensus.cc:374] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:15.513852 9059 raft_consensus.cc:729] T fb0794d2a14b4971bea153dd3d582569 P 4e45a2d12600427c9add4b3d9919293d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e45a2d12600427c9add4b3d9919293d, State: Initialized, Role: FOLLOWER
I20260811 05:13:15.513888 9057 raft_consensus.cc:729] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9ff216553e04ca8b5ae099e835a9a94, State: Initialized, Role: FOLLOWER
I20260811 05:13:15.514328 9059 consensus_queue.cc:260] T fb0794d2a14b4971bea153dd3d582569 P 4e45a2d12600427c9add4b3d9919293d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84db6206adf42b4b0983f845ae33d25" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44017 } } peers { permanent_uuid: "4e45a2d12600427c9add4b3d9919293d" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 41963 } } peers { permanent_uuid: "f9ff216553e04ca8b5ae099e835a9a94" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43357 } }
I20260811 05:13:15.514328 9057 consensus_queue.cc:260] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84db6206adf42b4b0983f845ae33d25" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44017 } } peers { permanent_uuid: "4e45a2d12600427c9add4b3d9919293d" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 41963 } } peers { permanent_uuid: "f9ff216553e04ca8b5ae099e835a9a94" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43357 } }
I20260811 05:13:15.515286 8605 heartbeater.cc:503] Master 127.7.67.190:41845 was elected leader, sending a full tablet report...
I20260811 05:13:15.515372 9057 ts_tablet_manager.cc:1428] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 05:13:15.515372 9059 ts_tablet_manager.cc:1428] T fb0794d2a14b4971bea153dd3d582569 P 4e45a2d12600427c9add4b3d9919293d: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 05:13:15.516446 9036 heartbeater.cc:503] Master 127.7.67.190:41845 was elected leader, sending a full tablet report...
I20260811 05:13:15.517258 9056 raft_consensus.cc:348] T fb0794d2a14b4971bea153dd3d582569 P c84db6206adf42b4b0983f845ae33d25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84db6206adf42b4b0983f845ae33d25" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44017 } } peers { permanent_uuid: "4e45a2d12600427c9add4b3d9919293d" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 41963 } } peers { permanent_uuid: "f9ff216553e04ca8b5ae099e835a9a94" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43357 } }
I20260811 05:13:15.517534 9056 raft_consensus.cc:374] T fb0794d2a14b4971bea153dd3d582569 P c84db6206adf42b4b0983f845ae33d25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:15.517617 9056 raft_consensus.cc:729] T fb0794d2a14b4971bea153dd3d582569 P c84db6206adf42b4b0983f845ae33d25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c84db6206adf42b4b0983f845ae33d25, State: Initialized, Role: FOLLOWER
I20260811 05:13:15.518070 9056 consensus_queue.cc:260] T fb0794d2a14b4971bea153dd3d582569 P c84db6206adf42b4b0983f845ae33d25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84db6206adf42b4b0983f845ae33d25" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44017 } } peers { permanent_uuid: "4e45a2d12600427c9add4b3d9919293d" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 41963 } } peers { permanent_uuid: "f9ff216553e04ca8b5ae099e835a9a94" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43357 } }
I20260811 05:13:15.518934 8824 heartbeater.cc:503] Master 127.7.67.190:41845 was elected leader, sending a full tablet report...
I20260811 05:13:15.519047 9056 ts_tablet_manager.cc:1428] T fb0794d2a14b4971bea153dd3d582569 P c84db6206adf42b4b0983f845ae33d25: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260811 05:13:15.687366 9037 tablet.cc:2369] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:13:15.712688 8825 tablet.cc:2369] T fb0794d2a14b4971bea153dd3d582569 P c84db6206adf42b4b0983f845ae33d25: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:13:15.730149 8606 tablet.cc:2369] T fb0794d2a14b4971bea153dd3d582569 P 4e45a2d12600427c9add4b3d9919293d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:13:15.763366 9066 raft_consensus.cc:482] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:15.763612 9066 raft_consensus.cc:504] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84db6206adf42b4b0983f845ae33d25" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44017 } } peers { permanent_uuid: "4e45a2d12600427c9add4b3d9919293d" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 41963 } } peers { permanent_uuid: "f9ff216553e04ca8b5ae099e835a9a94" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43357 } }
I20260811 05:13:15.764990 9066 leader_election.cc:288] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c84db6206adf42b4b0983f845ae33d25 (127.7.67.130:44017), 4e45a2d12600427c9add4b3d9919293d (127.7.67.129:41963)
I20260811 05:13:15.769253 9038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.765222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:44017 (local address 127.7.67.131:37951)
0811 05:13:15.765361 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:15.765380 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:15.765490 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:15.765883 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:15.765886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:15.765909 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:15.766092 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:15.766117 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.766873 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.766876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:15.767877 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:15.767887 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.768800 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.768804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:15.768910 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:15.768937 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 05:13:15.769016 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 05:13:15.769095 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":1}
I20260811 05:13:15.769318 9051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.765384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:37951 (local address 127.7.67.130:44017)
0811 05:13:15.765523 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:15.765531 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:15.765551 (+ 20us) server_negotiation.cc:407] Connection header received
0811 05:13:15.765615 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:15.765618 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:15.765700 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:15.765807 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:15.766252 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.766766 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.768022 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.768679 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.768776 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:15.768818 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:15.769039 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 05:13:15.769152 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 05:13:15.769861 9073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.765540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:41963 (local address 127.7.67.131:48749)
0811 05:13:15.765958 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:15.765971 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:15.766069 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:15.766367 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:15.766370 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:15.766387 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:15.766511 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:15.766516 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.767437 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.767439 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:15.768467 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:15.768478 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.769493 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.769497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:15.769591 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:15.769614 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 05:13:15.769713 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 05:13:15.769765 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":153,"threads_started":1}
I20260811 05:13:15.769933 9053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.765726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:48749 (local address 127.7.67.129:41963)
0811 05:13:15.765880 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:15.765885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:15.766030 (+ 145us) server_negotiation.cc:407] Connection header received
0811 05:13:15.766197 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:15.766201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:15.766267 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:15.766353 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:15.766625 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.767311 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.768618 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.769196 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.769331 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:15.769379 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:15.769737 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 05:13:15.769821 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 05:13:15.770066 8746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb0794d2a14b4971bea153dd3d582569" candidate_uuid: "f9ff216553e04ca8b5ae099e835a9a94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84db6206adf42b4b0983f845ae33d25" is_pre_election: true
I20260811 05:13:15.770571 8746 raft_consensus.cc:2454] T fb0794d2a14b4971bea153dd3d582569 P c84db6206adf42b4b0983f845ae33d25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9ff216553e04ca8b5ae099e835a9a94 in term 0.
I20260811 05:13:15.770656 8526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb0794d2a14b4971bea153dd3d582569" candidate_uuid: "f9ff216553e04ca8b5ae099e835a9a94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e45a2d12600427c9add4b3d9919293d" is_pre_election: true
I20260811 05:13:15.770952 8526 raft_consensus.cc:2454] T fb0794d2a14b4971bea153dd3d582569 P 4e45a2d12600427c9add4b3d9919293d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f9ff216553e04ca8b5ae099e835a9a94 in term 0.
I20260811 05:13:15.771189 8857 leader_election.cc:302] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c84db6206adf42b4b0983f845ae33d25, f9ff216553e04ca8b5ae099e835a9a94; no voters:
I20260811 05:13:15.771467 9066 raft_consensus.cc:2781] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:15.771567 9066 raft_consensus.cc:482] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:15.771631 9066 raft_consensus.cc:3037] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:15.772766 9066 raft_consensus.cc:504] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84db6206adf42b4b0983f845ae33d25" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44017 } } peers { permanent_uuid: "4e45a2d12600427c9add4b3d9919293d" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 41963 } } peers { permanent_uuid: "f9ff216553e04ca8b5ae099e835a9a94" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43357 } }
I20260811 05:13:15.773052 9066 leader_election.cc:288] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94 [CANDIDATE]: Term 1 election: Requested vote from peers c84db6206adf42b4b0983f845ae33d25 (127.7.67.130:44017), 4e45a2d12600427c9add4b3d9919293d (127.7.67.129:41963)
I20260811 05:13:15.773659 8746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb0794d2a14b4971bea153dd3d582569" candidate_uuid: "f9ff216553e04ca8b5ae099e835a9a94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c84db6206adf42b4b0983f845ae33d25"
I20260811 05:13:15.773659 8526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb0794d2a14b4971bea153dd3d582569" candidate_uuid: "f9ff216553e04ca8b5ae099e835a9a94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e45a2d12600427c9add4b3d9919293d"
I20260811 05:13:15.773809 8746 raft_consensus.cc:3037] T fb0794d2a14b4971bea153dd3d582569 P c84db6206adf42b4b0983f845ae33d25 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:15.773810 8526 raft_consensus.cc:3037] T fb0794d2a14b4971bea153dd3d582569 P 4e45a2d12600427c9add4b3d9919293d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:15.775283 8526 raft_consensus.cc:2454] T fb0794d2a14b4971bea153dd3d582569 P 4e45a2d12600427c9add4b3d9919293d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9ff216553e04ca8b5ae099e835a9a94 in term 1.
I20260811 05:13:15.775283 8746 raft_consensus.cc:2454] T fb0794d2a14b4971bea153dd3d582569 P c84db6206adf42b4b0983f845ae33d25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f9ff216553e04ca8b5ae099e835a9a94 in term 1.
I20260811 05:13:15.775614 8857 leader_election.cc:302] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c84db6206adf42b4b0983f845ae33d25, f9ff216553e04ca8b5ae099e835a9a94; no voters:
I20260811 05:13:15.775806 9066 raft_consensus.cc:2781] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:15.776067 9066 raft_consensus.cc:686] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94 [term 1 LEADER]: Becoming Leader. State: Replica: f9ff216553e04ca8b5ae099e835a9a94, State: Running, Role: LEADER
I20260811 05:13:15.776435 9066 consensus_queue.cc:237] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84db6206adf42b4b0983f845ae33d25" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44017 } } peers { permanent_uuid: "4e45a2d12600427c9add4b3d9919293d" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 41963 } } peers { permanent_uuid: "f9ff216553e04ca8b5ae099e835a9a94" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43357 } }
I20260811 05:13:15.780062 8311 catalog_manager.cc:5495] T fb0794d2a14b4971bea153dd3d582569 P f9ff216553e04ca8b5ae099e835a9a94 reported cstate change: term changed from 0 to 1, leader changed from <none> to f9ff216553e04ca8b5ae099e835a9a94 (127.7.67.131). New cstate: current_term: 1 leader_uuid: "f9ff216553e04ca8b5ae099e835a9a94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c84db6206adf42b4b0983f845ae33d25" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44017 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e45a2d12600427c9add4b3d9919293d" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 41963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f9ff216553e04ca8b5ae099e835a9a94" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43357 } health_report { overall_health: HEALTHY } } }
Aug 11 05:13:15 dist-test-slave-04bb krb5kdc[8246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.129@KRBTEST.COM
I20260811 05:13:15.857005 9053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.847286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40906 (local address 127.7.67.129:41963)
0811 05:13:15.847485 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:15.847490 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:15.847576 (+ 86us) server_negotiation.cc:407] Connection header received
0811 05:13:15.847689 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:15.847691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:15.847737 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:15.847820 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:15.848826 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.849329 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.850203 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.850427 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.850505 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:15.853280 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:15.853295 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:15.853314 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:15.853339 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:15.855349 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:15.855754 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:15.855760 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:15.855761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:15.855846 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:15.856083 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:15.856085 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:15.856086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:15.856372 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:15.856607 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:15.856746 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 05:13:15.856849 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
Aug 11 05:13:15 dist-test-slave-04bb krb5kdc[8246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425195, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 05:13:15 dist-test-slave-04bb krb5kdc[8246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425195, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.67.129@KRBTEST.COM
I20260811 05:13:15.887001 9053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.877614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40922 (local address 127.7.67.129:41963)
0811 05:13:15.877806 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:15.877811 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:15.877956 (+ 145us) server_negotiation.cc:407] Connection header received
0811 05:13:15.878146 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:15.878149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:15.878194 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:15.878285 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:15.879017 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.879614 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.880470 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.880780 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.880903 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:15.883593 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:15.883600 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:15.883608 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:15.883636 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:15.885302 (+ 1666us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:15.885743 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:15.885747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:15.885748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:15.885825 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:15.886132 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:15.886135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:15.886136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:15.886334 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:15.886455 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:15.886707 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 05:13:15.886808 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
Aug 11 05:13:15 dist-test-slave-04bb krb5kdc[8246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 05:13:15 dist-test-slave-04bb krb5kdc[8246](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.129@KRBTEST.COM
I20260811 05:13:15.916702 9053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:15.907443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40934 (local address 127.7.67.129:41963)
0811 05:13:15.907633 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:15.907639 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:15.907827 (+ 188us) server_negotiation.cc:407] Connection header received
0811 05:13:15.908025 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:15.908035 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:15.908085 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:15.908184 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:15.909010 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.909545 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.910445 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:15.910677 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:15.910799 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:15.913465 (+ 2666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:15.913468 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:15.913470 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:15.913495 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:15.914974 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:15.915407 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:15.915411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:15.915412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:15.915493 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:15.915856 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:15.915859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:15.915860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:15.916036 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:15.916144 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:15.916373 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 05:13:15.916476 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 05:13:15.918497 8470 tablet_service.cc:2908] Rejecting scan request for tablet fb0794d2a14b4971bea153dd3d582569: Uninitialized: clean time has not yet been initialized
I20260811 05:13:15.920828 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 8409
I20260811 05:13:15.939713 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 8613
I20260811 05:13:15.953907 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 8834
I20260811 05:13:15.972205 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 8263
2026-08-11T05:13:15Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2861 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:13:16 dist-test-slave-04bb krb5kdc[9097](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:13:16 dist-test-slave-04bb krb5kdc[9097](info): set up 2 sockets
Aug 11 05:13:16 dist-test-slave-04bb krb5kdc[9097](info): commencing operation
krb5kdc: starting...
W20260811 05:13:17.244952 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:13:17 dist-test-slave-04bb krb5kdc[9097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:13:17.375761 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:41969
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:40371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:41969
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:17.488067 9116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:17.488320 9116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:17.488359 9116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:17.492264 9116 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:17.492316 9116 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 05:13:17.492333 9116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:17.492352 9116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:17.492367 9116 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:17.492383 9116 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:17.497488 9116 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:40371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:41969
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:41969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:17.498456 9116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:17.499409 9116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:17.508028 9124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:17.508021 9127 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:17.508080 9125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:17.508373 9116 server_base.cc:1034] running on GCE node
I20260811 05:13:17.508984 9116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:17.510057 9116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:17.511251 9116 hybrid_clock.cc:648] HybridClock initialized: now 1786425197511236 us; error 26 us; skew 500 ppm
Aug 11 05:13:17 dist-test-slave-04bb krb5kdc[9097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425197, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:17.514562 9116 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:13:17.515655 9116 webserver.cc:460] Webserver started at http://127.7.67.190:34333/ using document root <none> and password file <none>
I20260811 05:13:17.516291 9116 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:17.516361 9116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:17.516673 9116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:17.518911 9116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "40a9f150d52c4cca83a073997534cd55"
format_stamp: "Formatted at 2026-08-11 05:13:17 on dist-test-slave-04bb"
server_key: "5c72ac2b5f7ba8bb6e5ef34303c879f4"
server_key_iv: "7d22b4cace181b44584ce96184258a71"
server_key_version: "encryptionkey@0"
I20260811 05:13:17.519542 9116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "40a9f150d52c4cca83a073997534cd55"
format_stamp: "Formatted at 2026-08-11 05:13:17 on dist-test-slave-04bb"
server_key: "5c72ac2b5f7ba8bb6e5ef34303c879f4"
server_key_iv: "7d22b4cace181b44584ce96184258a71"
server_key_version: "encryptionkey@0"
I20260811 05:13:17.523686 9116 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 05:13:17.526275 9138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:17.527585 9116 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:13:17.527742 9116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "40a9f150d52c4cca83a073997534cd55"
format_stamp: "Formatted at 2026-08-11 05:13:17 on dist-test-slave-04bb"
server_key: "5c72ac2b5f7ba8bb6e5ef34303c879f4"
server_key_iv: "7d22b4cace181b44584ce96184258a71"
server_key_version: "encryptionkey@0"
I20260811 05:13:17.527861 9116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:17.576912 9116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:17.580855 9116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:17.581126 9116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:17.590478 9116 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:41969
I20260811 05:13:17.590509 9239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:41969 every 8 connection(s)
I20260811 05:13:17.591495 9116 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:17.592324 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 9116
I20260811 05:13:17.592443 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:13:17.592769 7438 external_mini_cluster.cc:1428] Setting key 76588601755182914474d96929e253de
I20260811 05:13:17.594987 9240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:17.600407 9240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55: Bootstrap starting.
Aug 11 05:13:17 dist-test-slave-04bb krb5kdc[9097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:17.602873 9240 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:17.603961 9240 log.cc:824] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:17.605957 9240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55: No bootstrap required, opened a new log
I20260811 05:13:17.608117 9244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:17.594263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55476 (local address 127.7.67.190:41969)
0811 05:13:17.594731 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:17.594747 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:17.594786 (+ 39us) server_negotiation.cc:407] Connection header received
0811 05:13:17.595529 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:17.595548 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:17.595902 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:17.596263 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:17.597181 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:17.598274 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:17.599230 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:17.599749 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:17.599938 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:17.602772 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:17.602778 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:17.602793 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:17.602829 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:17.605036 (+ 2207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:17.605563 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:17.605568 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:17.605573 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:17.605696 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:17.606014 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:17.606019 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:17.606021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:17.606507 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:17.606725 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:17.607057 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 05:13:17.607362 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":131,"threads_started":1}
I20260811 05:13:17.609133 9240 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a9f150d52c4cca83a073997534cd55" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 41969 } }
I20260811 05:13:17.609370 9240 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:17.609450 9240 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40a9f150d52c4cca83a073997534cd55, State: Initialized, Role: FOLLOWER
I20260811 05:13:17.609879 9240 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a9f150d52c4cca83a073997534cd55" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 41969 } }
I20260811 05:13:17.610024 9240 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:17.610109 9240 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:17.610232 9240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:17.611148 9240 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a9f150d52c4cca83a073997534cd55" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 41969 } }
I20260811 05:13:17.611480 9240 leader_election.cc:302] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40a9f150d52c4cca83a073997534cd55; no voters:
I20260811 05:13:17.611802 9240 leader_election.cc:288] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:17.611922 9247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:17.612190 9247 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55 [term 1 LEADER]: Becoming Leader. State: Replica: 40a9f150d52c4cca83a073997534cd55, State: Running, Role: LEADER
I20260811 05:13:17.612519 9247 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a9f150d52c4cca83a073997534cd55" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 41969 } }
I20260811 05:13:17.613068 9240 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:17.614326 9248 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40a9f150d52c4cca83a073997534cd55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a9f150d52c4cca83a073997534cd55" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 41969 } } }
I20260811 05:13:17.614465 9248 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:17.614886 9249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40a9f150d52c4cca83a073997534cd55. Latest consensus state: current_term: 1 leader_uuid: "40a9f150d52c4cca83a073997534cd55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40a9f150d52c4cca83a073997534cd55" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 41969 } } }
I20260811 05:13:17.614996 9249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55 [sys.catalog]: This master's current role is: LEADER
W20260811 05:13:17.618305 9270 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:13:17.618402 9270 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:13:17.618489 9264 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:17.619284 9264 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:17.625082 9264 catalog_manager.cc:1347] Generated new cluster ID: 9e6581aa17f54fa0a333483c1f908de1
I20260811 05:13:17.625141 9264 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:17.638255 9264 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:17.639459 9264 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:17.647099 9264 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55: Generated new TSK 0
I20260811 05:13:17.647855 9264 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
I20260811 05:13:17.685545 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:41969
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:40371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:17.798220 9279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:17.798493 9279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:17.798571 9279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:17.802621 9279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:17.802702 9279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:17.802785 9279 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:13:17.802863 9279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:13:17.807957 9279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:40371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.67.190:41969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:17.809041 9279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:17.810153 9279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:17.820253 9290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:17.820341 9279 server_base.cc:1034] running on GCE node
W20260811 05:13:17.820245 9292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:17.820253 9288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:17.820866 9279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:17.821592 9279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:17.822794 9279 hybrid_clock.cc:648] HybridClock initialized: now 1786425197822760 us; error 45 us; skew 500 ppm
Aug 11 05:13:17 dist-test-slave-04bb krb5kdc[9097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425197, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:17.825579 9279 init.cc:377] Logged in from keytab as kudu/127.7.67.129@KRBTEST.COM (short username kudu)
I20260811 05:13:17.826370 9279 webserver.cc:460] Webserver started at http://127.7.67.129:37471/ using document root <none> and password file <none>
I20260811 05:13:17.826892 9279 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:17.826947 9279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:17.827101 9279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:17.829241 9279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "46601d56536e4c439c93bad01b8929cf"
format_stamp: "Formatted at 2026-08-11 05:13:17 on dist-test-slave-04bb"
server_key: "f7dfb26c719b96f632a3136fc58a0ffe"
server_key_iv: "5c9d9e4601da3f49cd1c087fc4042aa4"
server_key_version: "encryptionkey@0"
I20260811 05:13:17.829748 9279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "46601d56536e4c439c93bad01b8929cf"
format_stamp: "Formatted at 2026-08-11 05:13:17 on dist-test-slave-04bb"
server_key: "f7dfb26c719b96f632a3136fc58a0ffe"
server_key_iv: "5c9d9e4601da3f49cd1c087fc4042aa4"
server_key_version: "encryptionkey@0"
I20260811 05:13:17.833602 9279 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 05:13:17.836046 9302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:17.837455 9279 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:13:17.837594 9279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "46601d56536e4c439c93bad01b8929cf"
format_stamp: "Formatted at 2026-08-11 05:13:17 on dist-test-slave-04bb"
server_key: "f7dfb26c719b96f632a3136fc58a0ffe"
server_key_iv: "5c9d9e4601da3f49cd1c087fc4042aa4"
server_key_version: "encryptionkey@0"
I20260811 05:13:17.837703 9279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:17.852736 9279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:17.855693 9279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:17.855897 9279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:17.857475 9279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:17.857545 9279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:17.857613 9279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:17.857671 9279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 05:13:17 dist-test-slave-04bb krb5kdc[9097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425197, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:17.868860 9279 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:39569
I20260811 05:13:17.868955 9487 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:39569 every 8 connection(s)
I20260811 05:13:17.869854 9279 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:17.872288 9244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:17.858258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:39739 (local address 127.7.67.190:41969)
0811 05:13:17.858408 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:17.858412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:17.860214 (+ 1802us) server_negotiation.cc:407] Connection header received
0811 05:13:17.861563 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:17.861567 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:17.861629 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:17.861738 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:17.863240 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:17.863790 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:17.865193 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:17.865436 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:17.865553 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:17.868100 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:17.868103 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:17.868105 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:17.868131 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:17.869839 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:17.870552 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:17.870557 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:17.870561 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:17.870684 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:17.871075 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:17.871078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:17.871080 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:17.871307 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:17.871402 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:17.871896 (+ 494us) server_negotiation.cc:299] Negotiation successful
0811 05:13:17.872010 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 05:13:17.872324 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 9279
I20260811 05:13:17.872426 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:13:17.872768 7438 external_mini_cluster.cc:1428] Setting key ddf598465bb1bcdc18893945efa025d4
I20260811 05:13:17.872936 9318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:17.859361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:41969 (local address 127.7.67.129:39739)
0811 05:13:17.860034 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:17.860096 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:17.861343 (+ 1247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:17.861940 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:17.861950 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:17.862449 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:17.863077 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:17.863092 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:17.863923 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:17.863927 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:17.865054 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:17.865065 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:17.865597 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:17.865601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:17.865793 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:17.866407 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:17.866424 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:17.867922 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:17.870010 (+ 2088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:17.870017 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:17.870033 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:17.870390 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:17.870806 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:17.870811 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:17.870813 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:17.870969 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:17.871417 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:17.871422 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:17.871639 (+ 217us) client_negotiation.cc:769] Sending connection context
0811 05:13:17.871801 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 05:13:17.872038 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":148,"threads_started":1}
I20260811 05:13:17.874410 9488 heartbeater.cc:347] Connected to a master server at 127.7.67.190:41969
I20260811 05:13:17.874567 9488 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:17.875120 9488 heartbeater.cc:511] Master 127.7.67.190:41969 requested a full tablet report, sending...
I20260811 05:13:17.876763 9169 ts_manager.cc:194] Registered new tserver with Master: 46601d56536e4c439c93bad01b8929cf (127.7.67.129:39569)
I20260811 05:13:17.878127 9169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.129@KRBTEST.COM'} at 127.7.67.129:39739
WARNING: no policy specified for kudu/127.7.67.130@KRBTEST.COM; defaulting to no policy
I20260811 05:13:17.884722 9244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:17.877058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:34951 (local address 127.7.67.190:41969)
0811 05:13:17.877173 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:17.877178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:17.877598 (+ 420us) server_negotiation.cc:407] Connection header received
0811 05:13:17.877701 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:17.877705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:17.877746 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:17.877825 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:17.878388 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:17.879119 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:17.879985 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:17.880227 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:17.880331 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:17.881537 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:17.881540 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:17.881543 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:17.881567 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:17.883038 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:17.883469 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:17.883473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:17.883474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:17.883556 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:17.883860 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:17.883864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:17.883865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:17.884043 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:17.884162 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:17.884417 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 05:13:17.884562 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 05:13:17.884733 9496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:17.877139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:41969 (local address 127.7.67.129:34951)
0811 05:13:17.877495 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:17.877511 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:17.877618 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:17.877863 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:17.877868 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:17.878096 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:17.878261 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:17.878273 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:17.879250 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:17.879253 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:17.879863 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:17.879872 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:17.880348 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:17.880352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:17.880462 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:17.881041 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:17.881058 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:17.881389 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:17.883152 (+ 1763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:17.883155 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:17.883158 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:17.883364 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:17.883668 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:17.883671 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:17.883673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:17.883763 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:17.884146 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:17.884149 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:17.884250 (+ 101us) client_negotiation.cc:769] Sending connection context
0811 05:13:17.884360 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 05:13:17.884510 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":3,"thread_start_us":119,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
I20260811 05:13:17.901765 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:41969
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:40371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:18.015612 9499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:18.015880 9499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:18.015924 9499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:18.019991 9499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:18.020046 9499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:18.020084 9499 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:13:18.020151 9499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:13:18.025411 9499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:40371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.67.190:41969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:18.026439 9499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:18.027539 9499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:18.037384 9512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:18.037361 9510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:18.037680 9499 server_base.cc:1034] running on GCE node
W20260811 05:13:18.037385 9509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:18.038188 9499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:18.038841 9499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:18.040053 9499 hybrid_clock.cc:648] HybridClock initialized: now 1786425198040034 us; error 34 us; skew 500 ppm
Aug 11 05:13:18 dist-test-slave-04bb krb5kdc[9097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425198, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:18.043041 9499 init.cc:377] Logged in from keytab as kudu/127.7.67.130@KRBTEST.COM (short username kudu)
I20260811 05:13:18.043910 9499 webserver.cc:460] Webserver started at http://127.7.67.130:40215/ using document root <none> and password file <none>
I20260811 05:13:18.044481 9499 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:18.044649 9499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:18.044901 9499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:18.046924 9499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "f4665ad87d3a47dcaabe060612421752"
format_stamp: "Formatted at 2026-08-11 05:13:18 on dist-test-slave-04bb"
server_key: "6f075b4d7555f966f4ed9c7a1e9c71c8"
server_key_iv: "fdf12e92b642590dc5329c0034d933ec"
server_key_version: "encryptionkey@0"
I20260811 05:13:18.047506 9499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "f4665ad87d3a47dcaabe060612421752"
format_stamp: "Formatted at 2026-08-11 05:13:18 on dist-test-slave-04bb"
server_key: "6f075b4d7555f966f4ed9c7a1e9c71c8"
server_key_iv: "fdf12e92b642590dc5329c0034d933ec"
server_key_version: "encryptionkey@0"
I20260811 05:13:18.051319 9499 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 05:13:18.053930 9526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:18.055246 9499 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:18.055389 9499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "f4665ad87d3a47dcaabe060612421752"
format_stamp: "Formatted at 2026-08-11 05:13:18 on dist-test-slave-04bb"
server_key: "6f075b4d7555f966f4ed9c7a1e9c71c8"
server_key_iv: "fdf12e92b642590dc5329c0034d933ec"
server_key_version: "encryptionkey@0"
I20260811 05:13:18.055498 9499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:18.075037 9499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:18.078270 9499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:18.078467 9499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:18.079891 9499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:18.079962 9499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:18.080029 9499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:18.080075 9499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 05:13:18 dist-test-slave-04bb krb5kdc[9097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425198, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:18.092578 9499 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:44051
I20260811 05:13:18.093307 9714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:44051 every 8 connection(s)
I20260811 05:13:18.093920 9499 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:18.096046 9244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.081045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:56709 (local address 127.7.67.190:41969)
0811 05:13:18.081218 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:18.081222 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:18.082224 (+ 1002us) server_negotiation.cc:407] Connection header received
0811 05:13:18.083330 (+ 1106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:18.083333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:18.083402 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:18.083496 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:18.084979 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.085609 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.087107 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.087337 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.087454 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.090608 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:18.090612 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:18.090615 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:18.090649 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:18.092981 (+ 2332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:18.093841 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:18.093846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:18.093850 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:18.093973 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:18.094468 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:18.094471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:18.094473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:18.094899 (+ 426us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:18.095021 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:18.095731 (+ 710us) server_negotiation.cc:299] Negotiation successful
0811 05:13:18.095871 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 05:13:18.097010 9541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.081555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:41969 (local address 127.7.67.130:56709)
0811 05:13:18.082064 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:18.082106 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:18.083068 (+ 962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:18.083682 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:18.083694 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:18.084152 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:18.084822 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.084845 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.085755 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.085761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.086976 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.086986 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.087510 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.087514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.087717 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.088489 (+ 772us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:18.088516 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:18.090427 (+ 1911us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:18.093136 (+ 2709us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:18.093145 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:18.093162 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:18.093710 (+ 548us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:18.094104 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:18.094109 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:18.094114 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:18.094344 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:18.095028 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:18.095036 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:18.095421 (+ 385us) client_negotiation.cc:769] Sending connection context
0811 05:13:18.095669 (+ 248us) client_negotiation.cc:240] Negotiation successful
0811 05:13:18.095988 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":144,"threads_started":1}
I20260811 05:13:18.098636 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 9499
I20260811 05:13:18.098757 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:13:18.099128 7438 external_mini_cluster.cc:1428] Setting key 452d71675f7fd34cdec7b65034b65be2
I20260811 05:13:18.100639 9715 heartbeater.cc:347] Connected to a master server at 127.7.67.190:41969
I20260811 05:13:18.100859 9715 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:18.101428 9715 heartbeater.cc:511] Master 127.7.67.190:41969 requested a full tablet report, sending...
I20260811 05:13:18.102528 9166 ts_manager.cc:194] Registered new tserver with Master: f4665ad87d3a47dcaabe060612421752 (127.7.67.130:44051)
I20260811 05:13:18.103114 9166 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.130@KRBTEST.COM'} at 127.7.67.130:56709
I20260811 05:13:18.111907 9244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.100921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:35559 (local address 127.7.67.190:41969)
0811 05:13:18.103449 (+ 2528us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:18.103455 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:18.103474 (+ 19us) server_negotiation.cc:407] Connection header received
0811 05:13:18.103520 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:18.103524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:18.103577 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:18.103699 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:18.104314 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.105278 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.106687 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.106911 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.107016 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.108564 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:18.108570 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:18.108573 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:18.108601 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:18.110040 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:18.110619 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:18.110621 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:18.110622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:18.110713 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:18.111063 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:18.111065 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:18.111066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:18.111273 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:18.111355 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:18.111706 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 05:13:18.111797 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2434}
I20260811 05:13:18.112165 9724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.100776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:41969 (local address 127.7.67.130:35559)
0811 05:13:18.102313 (+ 1537us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:18.102330 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:18.102433 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:18.103724 (+ 1291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:18.103728 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:18.103987 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:18.104153 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.104165 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.105480 (+ 1315us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.105487 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.106544 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.106558 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.107041 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.107048 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.107195 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.107804 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:18.107826 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:18.108369 (+ 543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:18.110166 (+ 1797us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:18.110174 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:18.110178 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:18.110504 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:18.110820 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:18.110827 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:18.110831 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:18.110952 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:18.111386 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:18.111392 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:18.111507 (+ 115us) client_negotiation.cc:769] Sending connection context
0811 05:13:18.111632 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 05:13:18.111800 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1432,"thread_start_us":105,"threads_started":1}
WARNING: no policy specified for kudu/127.7.67.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
I20260811 05:13:18.133499 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:41969
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:40371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:18.252494 9729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:18.252859 9729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:18.252944 9729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:18.256981 9729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:18.257042 9729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:18.257115 9729 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:13:18.257190 9729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:13:18.262669 9729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:40371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.67.190:41969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:18.263744 9729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:18.264864 9729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:18.274757 9739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:18.274812 9742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:18.275007 9729 server_base.cc:1034] running on GCE node
W20260811 05:13:18.274767 9747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:18.275740 9729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:18.276352 9729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:18.277520 9729 hybrid_clock.cc:648] HybridClock initialized: now 1786425198277504 us; error 35 us; skew 500 ppm
Aug 11 05:13:18 dist-test-slave-04bb krb5kdc[9097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425198, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:18.280304 9729 init.cc:377] Logged in from keytab as kudu/127.7.67.131@KRBTEST.COM (short username kudu)
I20260811 05:13:18.281244 9729 webserver.cc:460] Webserver started at http://127.7.67.131:42533/ using document root <none> and password file <none>
I20260811 05:13:18.281819 9729 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:18.281924 9729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:18.282162 9729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:18.285073 9729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "a65c75fd6fe5471c87c098ddb89930e2"
format_stamp: "Formatted at 2026-08-11 05:13:18 on dist-test-slave-04bb"
server_key: "8e6ad8252d833e7716f46b596eaf5fac"
server_key_iv: "4319fd11252e155f2b6a9160d586fab5"
server_key_version: "encryptionkey@0"
I20260811 05:13:18.285794 9729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "a65c75fd6fe5471c87c098ddb89930e2"
format_stamp: "Formatted at 2026-08-11 05:13:18 on dist-test-slave-04bb"
server_key: "8e6ad8252d833e7716f46b596eaf5fac"
server_key_iv: "4319fd11252e155f2b6a9160d586fab5"
server_key_version: "encryptionkey@0"
I20260811 05:13:18.289544 9729 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 05:13:18.291957 9756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:18.293238 9729 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:13:18.293403 9729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "a65c75fd6fe5471c87c098ddb89930e2"
format_stamp: "Formatted at 2026-08-11 05:13:18 on dist-test-slave-04bb"
server_key: "8e6ad8252d833e7716f46b596eaf5fac"
server_key_iv: "4319fd11252e155f2b6a9160d586fab5"
server_key_version: "encryptionkey@0"
I20260811 05:13:18.293510 9729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:18.307657 9729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:18.310590 9729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:18.310791 9729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:18.312299 9729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:18.312372 9729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:18.312438 9729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:18.312491 9729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 05:13:18 dist-test-slave-04bb krb5kdc[9097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425198, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:18.323477 9729 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:43547
I20260811 05:13:18.324339 9729 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:13:18.324626 9963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:43547 every 8 connection(s)
I20260811 05:13:18.327386 9244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.312977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:55175 (local address 127.7.67.190:41969)
0811 05:13:18.313145 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:18.313152 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:18.314593 (+ 1441us) server_negotiation.cc:407] Connection header received
0811 05:13:18.315786 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:18.315793 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:18.315866 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:18.315971 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:18.317825 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.318731 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.320622 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.320855 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.320942 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.323953 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:18.323957 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:18.323959 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:18.323994 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:18.325593 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:18.326047 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:18.326050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:18.326051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:18.326129 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:18.326435 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:18.326437 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:18.326438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:18.326604 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:18.326680 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:18.327152 (+ 472us) server_negotiation.cc:299] Negotiation successful
0811 05:13:18.327265 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 05:13:18.328136 9774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.313976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:41969 (local address 127.7.67.131:55175)
0811 05:13:18.314441 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:18.314490 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:18.315552 (+ 1062us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:18.316294 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:18.316303 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:18.316848 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:18.317612 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.317634 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.318912 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.318918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.320457 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.320469 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.321021 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.321025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.321251 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.321985 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:18.322012 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:18.323785 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:18.325697 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:18.325701 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:18.325712 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:18.325953 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:18.326214 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:18.326216 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:18.326218 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:18.326353 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:18.326691 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:18.326696 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:18.326903 (+ 207us) client_negotiation.cc:769] Sending connection context
0811 05:13:18.327050 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 05:13:18.327287 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":143,"threads_started":1}
I20260811 05:13:18.329339 9965 heartbeater.cc:347] Connected to a master server at 127.7.67.190:41969
I20260811 05:13:18.329504 9965 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:18.329995 9965 heartbeater.cc:511] Master 127.7.67.190:41969 requested a full tablet report, sending...
I20260811 05:13:18.330116 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 9729
I20260811 05:13:18.330250 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:13:18.330583 7438 external_mini_cluster.cc:1428] Setting key a440f20f07a9145d3cde417344857586
I20260811 05:13:18.331291 9169 ts_manager.cc:194] Registered new tserver with Master: a65c75fd6fe5471c87c098ddb89930e2 (127.7.67.131:43547)
I20260811 05:13:18.331975 9169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.131@KRBTEST.COM'} at 127.7.67.131:55175
I20260811 05:13:18.332895 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:13:18.339825 9975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.331781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:41969 (local address 127.7.67.131:34583)
0811 05:13:18.332210 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:18.332234 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:18.332319 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:18.332898 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:18.332902 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:18.333112 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:18.333223 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.333231 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.334211 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.334214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.334746 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.334754 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.335367 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.335373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.335493 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.335955 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:18.335974 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:18.336398 (+ 424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:18.338101 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:18.338107 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:18.338112 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:18.338458 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:18.338846 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:18.338851 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:18.338856 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:18.338960 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:18.339318 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:18.339322 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:18.339424 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 05:13:18.339544 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 05:13:18.339699 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":125,"threads_started":1}
I20260811 05:13:18.339938 9244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.331983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:34583 (local address 127.7.67.190:41969)
0811 05:13:18.332686 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:18.332690 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:18.332703 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:13:18.332733 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:18.332738 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:18.332782 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:18.332877 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:18.333359 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.334091 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.334862 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.335237 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.335351 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.336587 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:18.336591 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:18.336593 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:18.336618 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:18.337976 (+ 1358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:18.338643 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:18.338646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:18.338647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:18.338744 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:18.339060 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:18.339062 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:18.339063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:18.339215 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:18.339291 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:18.339646 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 05:13:18.339773 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631}
Aug 11 05:13:18 dist-test-slave-04bb krb5kdc[9097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425198, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 05:13:18 dist-test-slave-04bb krb5kdc[9097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425198, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:18.360917 9244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.353118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55492 (local address 127.7.67.190:41969)
0811 05:13:18.353239 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:18.353243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:18.353258 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:13:18.353295 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:18.353297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:18.353333 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:18.353402 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:18.354036 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.354513 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.355294 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.355496 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.355576 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.357852 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:18.357855 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:18.357857 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:18.357879 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:18.359400 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:18.359859 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:18.359862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:18.359863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:18.359937 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:18.360168 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:18.360170 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:18.360171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:18.360361 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:18.360464 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:18.360713 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 05:13:18.360804 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 05:13:18.363770 9166 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55492:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:13:18.366848 9166 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:18.382648 9995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.376139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57238 (local address 127.7.67.130:44051)
0811 05:13:18.376943 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:18.376956 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:18.377502 (+ 546us) server_negotiation.cc:407] Connection header received
0811 05:13:18.377776 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:18.377782 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:18.377930 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:18.378061 (+ 131us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:18.378524 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.379335 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.381012 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.381863 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.381990 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.382186 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:18.382399 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 05:13:18.382501 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":230,"threads_started":1}
I20260811 05:13:18.382637 9994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.375963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:44051 (local address 127.0.0.1:57238)
0811 05:13:18.377391 (+ 1428us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:18.377410 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:18.377593 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:18.378205 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:18.378208 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:18.378225 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:18.378322 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.378327 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.379540 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.379545 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.380869 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.380881 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.382006 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.382013 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.382145 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.382190 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 05:13:18.382356 (+ 166us) client_negotiation.cc:240] Negotiation successful
0811 05:13:18.382485 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1051,"mutex_wait_us":2,"thread_start_us":79,"threads_started":1}
I20260811 05:13:18.383381 10000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.377245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58452 (local address 127.7.67.129:39569)
0811 05:13:18.377680 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:18.377690 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:18.377714 (+ 24us) server_negotiation.cc:407] Connection header received
0811 05:13:18.377797 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:18.377801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:18.377954 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:18.378127 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:18.378544 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.379658 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.381431 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.382062 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.382163 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.382298 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:18.383109 (+ 811us) server_negotiation.cc:299] Negotiation successful
0811 05:13:18.383236 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":100,"threads_started":1}
I20260811 05:13:18.383734 9999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.376748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:39569 (local address 127.0.0.1:58452)
0811 05:13:18.377314 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:18.377328 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:18.377460 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:18.378279 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:18.378282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:18.378298 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:18.378414 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.378420 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.379803 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.379807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.381282 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.381294 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.382284 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.382289 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.382402 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.382428 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 05:13:18.382504 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 05:13:18.382590 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":496,"mutex_wait_us":158,"thread_start_us":112,"threads_started":1}
I20260811 05:13:18.384011 9993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.376027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:43547 (local address 127.0.0.1:36788)
0811 05:13:18.377217 (+ 1190us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:18.377268 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:18.377460 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:18.378083 (+ 623us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:18.378089 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:18.378115 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:18.378289 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.378298 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.379803 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.379807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.381282 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.381294 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.382677 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.382682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.382873 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.383660 (+ 787us) client_negotiation.cc:769] Sending connection context
0811 05:13:18.383754 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 05:13:18.383846 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1020,"mutex_wait_us":136,"spinlock_wait_cycles":21504,"thread_start_us":93,"threads_started":1}
I20260811 05:13:18.384328 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.376435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36788 (local address 127.7.67.131:43547)
0811 05:13:18.376772 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:18.376786 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:18.377367 (+ 581us) server_negotiation.cc:407] Connection header received
0811 05:13:18.377653 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:18.377679 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:18.377892 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:18.378061 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:18.378422 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.379658 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.381428 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.382310 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.382419 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.382592 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:18.383953 (+ 1361us) server_negotiation.cc:299] Negotiation successful
0811 05:13:18.384187 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":147,"threads_started":1}
I20260811 05:13:18.386097 9609 tablet_service.cc:1467] Processing CreateTablet for tablet 8e40bef190f8418688a6eae0fbfb9779 (DEFAULT_TABLE table=test-table [id=77e78ff90316459382641fe35ad15894]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:18.386344 9388 tablet_service.cc:1467] Processing CreateTablet for tablet 8e40bef190f8418688a6eae0fbfb9779 (DEFAULT_TABLE table=test-table [id=77e78ff90316459382641fe35ad15894]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:18.387339 9609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e40bef190f8418688a6eae0fbfb9779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:18.386989 9851 tablet_service.cc:1467] Processing CreateTablet for tablet 8e40bef190f8418688a6eae0fbfb9779 (DEFAULT_TABLE table=test-table [id=77e78ff90316459382641fe35ad15894]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:18.387526 9388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e40bef190f8418688a6eae0fbfb9779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:18.387931 9851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e40bef190f8418688a6eae0fbfb9779. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:18.394079 10001 tablet_bootstrap.cc:492] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf: Bootstrap starting.
I20260811 05:13:18.396281 10004 tablet_bootstrap.cc:492] T 8e40bef190f8418688a6eae0fbfb9779 P a65c75fd6fe5471c87c098ddb89930e2: Bootstrap starting.
I20260811 05:13:18.396703 10003 tablet_bootstrap.cc:492] T 8e40bef190f8418688a6eae0fbfb9779 P f4665ad87d3a47dcaabe060612421752: Bootstrap starting.
I20260811 05:13:18.396812 10001 tablet_bootstrap.cc:654] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:18.398102 10001 log.cc:824] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:18.398916 10003 tablet_bootstrap.cc:654] T 8e40bef190f8418688a6eae0fbfb9779 P f4665ad87d3a47dcaabe060612421752: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:18.398869 10004 tablet_bootstrap.cc:654] T 8e40bef190f8418688a6eae0fbfb9779 P a65c75fd6fe5471c87c098ddb89930e2: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:18.399583 10003 log.cc:824] T 8e40bef190f8418688a6eae0fbfb9779 P f4665ad87d3a47dcaabe060612421752: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:18.399806 10004 log.cc:824] T 8e40bef190f8418688a6eae0fbfb9779 P a65c75fd6fe5471c87c098ddb89930e2: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:18.400493 10001 tablet_bootstrap.cc:492] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf: No bootstrap required, opened a new log
I20260811 05:13:18.400739 10001 ts_tablet_manager.cc:1397] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.002s
I20260811 05:13:18.401762 10003 tablet_bootstrap.cc:492] T 8e40bef190f8418688a6eae0fbfb9779 P f4665ad87d3a47dcaabe060612421752: No bootstrap required, opened a new log
I20260811 05:13:18.401993 10003 ts_tablet_manager.cc:1397] T 8e40bef190f8418688a6eae0fbfb9779 P f4665ad87d3a47dcaabe060612421752: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 05:13:18.405560 10004 tablet_bootstrap.cc:492] T 8e40bef190f8418688a6eae0fbfb9779 P a65c75fd6fe5471c87c098ddb89930e2: No bootstrap required, opened a new log
I20260811 05:13:18.405457 10001 raft_consensus.cc:348] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } } peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } }
I20260811 05:13:18.405753 10001 raft_consensus.cc:374] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:18.405784 10004 ts_tablet_manager.cc:1397] T 8e40bef190f8418688a6eae0fbfb9779 P a65c75fd6fe5471c87c098ddb89930e2: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260811 05:13:18.405823 10001 raft_consensus.cc:729] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46601d56536e4c439c93bad01b8929cf, State: Initialized, Role: FOLLOWER
I20260811 05:13:18.406209 10003 raft_consensus.cc:348] T 8e40bef190f8418688a6eae0fbfb9779 P f4665ad87d3a47dcaabe060612421752 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } } peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } }
I20260811 05:13:18.406392 10001 consensus_queue.cc:260] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } } peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } }
I20260811 05:13:18.406491 10003 raft_consensus.cc:374] T 8e40bef190f8418688a6eae0fbfb9779 P f4665ad87d3a47dcaabe060612421752 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:18.406548 10003 raft_consensus.cc:729] T 8e40bef190f8418688a6eae0fbfb9779 P f4665ad87d3a47dcaabe060612421752 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4665ad87d3a47dcaabe060612421752, State: Initialized, Role: FOLLOWER
I20260811 05:13:18.407087 9488 heartbeater.cc:503] Master 127.7.67.190:41969 was elected leader, sending a full tablet report...
I20260811 05:13:18.407048 10003 consensus_queue.cc:260] T 8e40bef190f8418688a6eae0fbfb9779 P f4665ad87d3a47dcaabe060612421752 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } } peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } }
I20260811 05:13:18.407521 10001 ts_tablet_manager.cc:1428] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf: Time spent starting tablet: real 0.007s user 0.004s sys 0.002s
I20260811 05:13:18.408149 9715 heartbeater.cc:503] Master 127.7.67.190:41969 was elected leader, sending a full tablet report...
I20260811 05:13:18.408475 10003 ts_tablet_manager.cc:1428] T 8e40bef190f8418688a6eae0fbfb9779 P f4665ad87d3a47dcaabe060612421752: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 05:13:18.409530 10004 raft_consensus.cc:348] T 8e40bef190f8418688a6eae0fbfb9779 P a65c75fd6fe5471c87c098ddb89930e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } } peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } }
I20260811 05:13:18.409788 10004 raft_consensus.cc:374] T 8e40bef190f8418688a6eae0fbfb9779 P a65c75fd6fe5471c87c098ddb89930e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:18.409854 10004 raft_consensus.cc:729] T 8e40bef190f8418688a6eae0fbfb9779 P a65c75fd6fe5471c87c098ddb89930e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a65c75fd6fe5471c87c098ddb89930e2, State: Initialized, Role: FOLLOWER
I20260811 05:13:18.410286 10004 consensus_queue.cc:260] T 8e40bef190f8418688a6eae0fbfb9779 P a65c75fd6fe5471c87c098ddb89930e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } } peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } }
I20260811 05:13:18.411163 10004 ts_tablet_manager.cc:1428] T 8e40bef190f8418688a6eae0fbfb9779 P a65c75fd6fe5471c87c098ddb89930e2: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 05:13:18.411342 9965 heartbeater.cc:503] Master 127.7.67.190:41969 was elected leader, sending a full tablet report...
I20260811 05:13:18.433576 10010 raft_consensus.cc:482] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:18.433835 10010 raft_consensus.cc:504] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } } peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } }
I20260811 05:13:18.435091 10010 leader_election.cc:288] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4665ad87d3a47dcaabe060612421752 (127.7.67.130:44051), a65c75fd6fe5471c87c098ddb89930e2 (127.7.67.131:43547)
I20260811 05:13:18.439446 10019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.435373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:43547 (local address 127.7.67.129:53789)
0811 05:13:18.435800 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:18.435816 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:18.435962 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:18.436286 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:18.436289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:18.436311 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:18.436459 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.436467 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.437364 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.437366 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.438250 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.438261 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.439007 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.439013 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.439106 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.439144 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 05:13:18.439207 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 05:13:18.439260 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":59,"thread_start_us":128,"threads_started":1}
I20260811 05:13:18.439497 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.435576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:53789 (local address 127.7.67.131:43547)
0811 05:13:18.435802 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:18.435808 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:18.435936 (+ 128us) server_negotiation.cc:407] Connection header received
0811 05:13:18.436094 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:18.436100 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:18.436167 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:18.436260 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:18.436644 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.437258 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.438393 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.438854 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.439001 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.439058 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:18.439282 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 05:13:18.439365 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260811 05:13:18.439550 9995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.435509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:43457 (local address 127.7.67.130:44051)
0811 05:13:18.435694 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:18.435701 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:18.435997 (+ 296us) server_negotiation.cc:407] Connection header received
0811 05:13:18.436188 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:18.436191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:18.436257 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:18.436360 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:18.436670 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.437190 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.438416 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.438827 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.438915 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.438994 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:18.439351 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 05:13:18.439422 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 05:13:18.439622 10020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.435373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:44051 (local address 127.7.67.129:43457)
0811 05:13:18.435917 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:18.435929 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:18.436059 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:18.436392 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:18.436395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:18.436412 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:18.436559 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.436569 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.437312 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.437318 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.438266 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.438276 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.438954 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.438960 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.439070 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.439103 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 05:13:18.439326 (+ 223us) client_negotiation.cc:240] Negotiation successful
0811 05:13:18.439394 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"mutex_wait_us":64,"thread_start_us":111,"threads_started":1}
I20260811 05:13:18.440102 9879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e40bef190f8418688a6eae0fbfb9779" candidate_uuid: "46601d56536e4c439c93bad01b8929cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a65c75fd6fe5471c87c098ddb89930e2" is_pre_election: true
I20260811 05:13:18.440157 9640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e40bef190f8418688a6eae0fbfb9779" candidate_uuid: "46601d56536e4c439c93bad01b8929cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4665ad87d3a47dcaabe060612421752" is_pre_election: true
I20260811 05:13:18.440430 9879 raft_consensus.cc:2454] T 8e40bef190f8418688a6eae0fbfb9779 P a65c75fd6fe5471c87c098ddb89930e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46601d56536e4c439c93bad01b8929cf in term 0.
I20260811 05:13:18.440438 9640 raft_consensus.cc:2454] T 8e40bef190f8418688a6eae0fbfb9779 P f4665ad87d3a47dcaabe060612421752 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 46601d56536e4c439c93bad01b8929cf in term 0.
I20260811 05:13:18.440917 9310 leader_election.cc:302] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46601d56536e4c439c93bad01b8929cf, a65c75fd6fe5471c87c098ddb89930e2; no voters:
I20260811 05:13:18.441200 10010 raft_consensus.cc:2781] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:18.441294 10010 raft_consensus.cc:482] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:18.441326 10010 raft_consensus.cc:3037] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:18.442467 10010 raft_consensus.cc:504] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } } peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } }
I20260811 05:13:18.442855 10010 leader_election.cc:288] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf [CANDIDATE]: Term 1 election: Requested vote from peers f4665ad87d3a47dcaabe060612421752 (127.7.67.130:44051), a65c75fd6fe5471c87c098ddb89930e2 (127.7.67.131:43547)
I20260811 05:13:18.443207 9640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e40bef190f8418688a6eae0fbfb9779" candidate_uuid: "46601d56536e4c439c93bad01b8929cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4665ad87d3a47dcaabe060612421752"
I20260811 05:13:18.443216 9879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e40bef190f8418688a6eae0fbfb9779" candidate_uuid: "46601d56536e4c439c93bad01b8929cf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a65c75fd6fe5471c87c098ddb89930e2"
I20260811 05:13:18.443349 9640 raft_consensus.cc:3037] T 8e40bef190f8418688a6eae0fbfb9779 P f4665ad87d3a47dcaabe060612421752 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:18.443354 9879 raft_consensus.cc:3037] T 8e40bef190f8418688a6eae0fbfb9779 P a65c75fd6fe5471c87c098ddb89930e2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:18.444391 9879 raft_consensus.cc:2454] T 8e40bef190f8418688a6eae0fbfb9779 P a65c75fd6fe5471c87c098ddb89930e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46601d56536e4c439c93bad01b8929cf in term 1.
I20260811 05:13:18.444720 9310 leader_election.cc:302] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46601d56536e4c439c93bad01b8929cf, a65c75fd6fe5471c87c098ddb89930e2; no voters:
I20260811 05:13:18.444875 10010 raft_consensus.cc:2781] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:18.445096 9640 raft_consensus.cc:2454] T 8e40bef190f8418688a6eae0fbfb9779 P f4665ad87d3a47dcaabe060612421752 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 46601d56536e4c439c93bad01b8929cf in term 1.
I20260811 05:13:18.445127 10010 raft_consensus.cc:686] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf [term 1 LEADER]: Becoming Leader. State: Replica: 46601d56536e4c439c93bad01b8929cf, State: Running, Role: LEADER
I20260811 05:13:18.445469 10010 consensus_queue.cc:237] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } } peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } }
I20260811 05:13:18.448850 9169 catalog_manager.cc:5495] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf reported cstate change: term changed from 0 to 1, leader changed from <none> to 46601d56536e4c439c93bad01b8929cf (127.7.67.129). New cstate: current_term: 1 leader_uuid: "46601d56536e4c439c93bad01b8929cf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } health_report { overall_health: HEALTHY } } }
I20260811 05:13:18.466508 10000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.462719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58462 (local address 127.7.67.129:39569)
0811 05:13:18.462867 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:18.462871 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:18.462891 (+ 20us) server_negotiation.cc:407] Connection header received
0811 05:13:18.463108 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:18.463111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:18.463176 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:18.463303 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:18.463554 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.464052 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.465000 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.465232 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.465343 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.465663 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:18.466076 (+ 413us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:18.466203 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:18.466324 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 05:13:18.466377 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 05:13:18.472646 9640 raft_consensus.cc:1261] T 8e40bef190f8418688a6eae0fbfb9779 P f4665ad87d3a47dcaabe060612421752 [term 1 FOLLOWER]: Refusing update from remote peer 46601d56536e4c439c93bad01b8929cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:18.473415 10022 consensus_queue.cc:1035] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:18.474663 9879 raft_consensus.cc:1261] T 8e40bef190f8418688a6eae0fbfb9779 P a65c75fd6fe5471c87c098ddb89930e2 [term 1 FOLLOWER]: Refusing update from remote peer 46601d56536e4c439c93bad01b8929cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:18.475270 10010 consensus_queue.cc:1035] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf [LEADER]: Connected to new peer: Peer: permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:18.482894 10033 mvcc.cc:205] Tried to move back new op lower bound from 7317197612934979584 to 7317197612835635200. Current Snapshot: MvccSnapshot[applied={T|T < 7317197612934979584}]
I20260811 05:13:18.495312 9169 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:18.495497 9169 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:18.498252 9169 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55: Sending DeleteTablet for 3 replicas of tablet 8e40bef190f8418688a6eae0fbfb9779
I20260811 05:13:18.499125 9609 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e40bef190f8418688a6eae0fbfb9779 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:18 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:57238
I20260811 05:13:18.499109 9851 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e40bef190f8418688a6eae0fbfb9779 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:18 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:36788
I20260811 05:13:18.499284 9388 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e40bef190f8418688a6eae0fbfb9779 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:18 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:58452
I20260811 05:13:18.499693 10044 tablet_replica.cc:330] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf: stopping tablet replica
I20260811 05:13:18.499704 10041 tablet_replica.cc:330] T 8e40bef190f8418688a6eae0fbfb9779 P a65c75fd6fe5471c87c098ddb89930e2: stopping tablet replica
I20260811 05:13:18.500008 10044 raft_consensus.cc:2229] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:13:18.500015 10041 raft_consensus.cc:2229] T 8e40bef190f8418688a6eae0fbfb9779 P a65c75fd6fe5471c87c098ddb89930e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:18.500344 10041 raft_consensus.cc:2258] T 8e40bef190f8418688a6eae0fbfb9779 P a65c75fd6fe5471c87c098ddb89930e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:18.500376 10044 raft_consensus.cc:2258] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:18.500974 9169 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55492:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:13:18.501411 9169 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:18.501856 10044 ts_tablet_manager.cc:1905] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:18.501966 10041 ts_tablet_manager.cc:1905] T 8e40bef190f8418688a6eae0fbfb9779 P a65c75fd6fe5471c87c098ddb89930e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:18.502550 10040 tablet_replica.cc:330] T 8e40bef190f8418688a6eae0fbfb9779 P f4665ad87d3a47dcaabe060612421752: stopping tablet replica
I20260811 05:13:18.502938 10040 raft_consensus.cc:2229] T 8e40bef190f8418688a6eae0fbfb9779 P f4665ad87d3a47dcaabe060612421752 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:18.503304 10040 raft_consensus.cc:2258] T 8e40bef190f8418688a6eae0fbfb9779 P f4665ad87d3a47dcaabe060612421752 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:18.504858 10044 ts_tablet_manager.cc:1918] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:18.504952 10044 log.cc:1196] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-0/wal/wals/8e40bef190f8418688a6eae0fbfb9779
I20260811 05:13:18.505240 10044 ts_tablet_manager.cc:1939] T 8e40bef190f8418688a6eae0fbfb9779 P 46601d56536e4c439c93bad01b8929cf: Deleting consensus metadata
I20260811 05:13:18.506361 9143 catalog_manager.cc:4841] TS 46601d56536e4c439c93bad01b8929cf (127.7.67.129:39569): tablet 8e40bef190f8418688a6eae0fbfb9779 (table test-table [id=77e78ff90316459382641fe35ad15894]) successfully deleted
I20260811 05:13:18.507205 9388 tablet_service.cc:1467] Processing CreateTablet for tablet e7f858acf89e4211a1fca7ba3bb3fd32 (DEFAULT_TABLE table=test-table [id=4a6052d07e86429f8d34c786472ebc76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:18.507462 9388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7f858acf89e4211a1fca7ba3bb3fd32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:18.507746 10040 ts_tablet_manager.cc:1905] T 8e40bef190f8418688a6eae0fbfb9779 P f4665ad87d3a47dcaabe060612421752: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:18.508351 10041 ts_tablet_manager.cc:1918] T 8e40bef190f8418688a6eae0fbfb9779 P a65c75fd6fe5471c87c098ddb89930e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:18.508464 10041 log.cc:1196] T 8e40bef190f8418688a6eae0fbfb9779 P a65c75fd6fe5471c87c098ddb89930e2: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/8e40bef190f8418688a6eae0fbfb9779
I20260811 05:13:18.508865 10041 ts_tablet_manager.cc:1939] T 8e40bef190f8418688a6eae0fbfb9779 P a65c75fd6fe5471c87c098ddb89930e2: Deleting consensus metadata
I20260811 05:13:18.509845 10001 tablet_bootstrap.cc:492] T e7f858acf89e4211a1fca7ba3bb3fd32 P 46601d56536e4c439c93bad01b8929cf: Bootstrap starting.
I20260811 05:13:18.510828 10001 tablet_bootstrap.cc:654] T e7f858acf89e4211a1fca7ba3bb3fd32 P 46601d56536e4c439c93bad01b8929cf: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:18.510980 10040 ts_tablet_manager.cc:1918] T 8e40bef190f8418688a6eae0fbfb9779 P f4665ad87d3a47dcaabe060612421752: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:18.511111 10040 log.cc:1196] T 8e40bef190f8418688a6eae0fbfb9779 P f4665ad87d3a47dcaabe060612421752: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/8e40bef190f8418688a6eae0fbfb9779
I20260811 05:13:18.511533 10040 ts_tablet_manager.cc:1939] T 8e40bef190f8418688a6eae0fbfb9779 P f4665ad87d3a47dcaabe060612421752: Deleting consensus metadata
I20260811 05:13:18.511760 9851 tablet_service.cc:1467] Processing CreateTablet for tablet e7f858acf89e4211a1fca7ba3bb3fd32 (DEFAULT_TABLE table=test-table [id=4a6052d07e86429f8d34c786472ebc76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:18.511997 9851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7f858acf89e4211a1fca7ba3bb3fd32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:18.512156 10001 tablet_bootstrap.cc:492] T e7f858acf89e4211a1fca7ba3bb3fd32 P 46601d56536e4c439c93bad01b8929cf: No bootstrap required, opened a new log
I20260811 05:13:18.512243 10001 ts_tablet_manager.cc:1397] T e7f858acf89e4211a1fca7ba3bb3fd32 P 46601d56536e4c439c93bad01b8929cf: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260811 05:13:18.512815 10001 raft_consensus.cc:348] T e7f858acf89e4211a1fca7ba3bb3fd32 P 46601d56536e4c439c93bad01b8929cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } }
I20260811 05:13:18.512936 10001 raft_consensus.cc:374] T e7f858acf89e4211a1fca7ba3bb3fd32 P 46601d56536e4c439c93bad01b8929cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:18.512960 10001 raft_consensus.cc:729] T e7f858acf89e4211a1fca7ba3bb3fd32 P 46601d56536e4c439c93bad01b8929cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46601d56536e4c439c93bad01b8929cf, State: Initialized, Role: FOLLOWER
I20260811 05:13:18.513046 10001 consensus_queue.cc:260] T e7f858acf89e4211a1fca7ba3bb3fd32 P 46601d56536e4c439c93bad01b8929cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } }
I20260811 05:13:18.513815 10001 ts_tablet_manager.cc:1428] T e7f858acf89e4211a1fca7ba3bb3fd32 P 46601d56536e4c439c93bad01b8929cf: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:18.514161 10004 tablet_bootstrap.cc:492] T e7f858acf89e4211a1fca7ba3bb3fd32 P a65c75fd6fe5471c87c098ddb89930e2: Bootstrap starting.
I20260811 05:13:18.514328 9145 catalog_manager.cc:4841] TS a65c75fd6fe5471c87c098ddb89930e2 (127.7.67.131:43547): tablet 8e40bef190f8418688a6eae0fbfb9779 (table test-table [id=77e78ff90316459382641fe35ad15894]) successfully deleted
I20260811 05:13:18.515268 10004 tablet_bootstrap.cc:654] T e7f858acf89e4211a1fca7ba3bb3fd32 P a65c75fd6fe5471c87c098ddb89930e2: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:18.515450 9143 catalog_manager.cc:4841] TS f4665ad87d3a47dcaabe060612421752 (127.7.67.130:44051): tablet 8e40bef190f8418688a6eae0fbfb9779 (table test-table [id=77e78ff90316459382641fe35ad15894]) successfully deleted
I20260811 05:13:18.515933 9609 tablet_service.cc:1467] Processing CreateTablet for tablet e7f858acf89e4211a1fca7ba3bb3fd32 (DEFAULT_TABLE table=test-table [id=4a6052d07e86429f8d34c786472ebc76]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:18.516184 9609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e7f858acf89e4211a1fca7ba3bb3fd32. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:18.516635 10004 tablet_bootstrap.cc:492] T e7f858acf89e4211a1fca7ba3bb3fd32 P a65c75fd6fe5471c87c098ddb89930e2: No bootstrap required, opened a new log
I20260811 05:13:18.516733 10004 ts_tablet_manager.cc:1397] T e7f858acf89e4211a1fca7ba3bb3fd32 P a65c75fd6fe5471c87c098ddb89930e2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:13:18.517300 10004 raft_consensus.cc:348] T e7f858acf89e4211a1fca7ba3bb3fd32 P a65c75fd6fe5471c87c098ddb89930e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } }
I20260811 05:13:18.517437 10004 raft_consensus.cc:374] T e7f858acf89e4211a1fca7ba3bb3fd32 P a65c75fd6fe5471c87c098ddb89930e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:18.517483 10004 raft_consensus.cc:729] T e7f858acf89e4211a1fca7ba3bb3fd32 P a65c75fd6fe5471c87c098ddb89930e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a65c75fd6fe5471c87c098ddb89930e2, State: Initialized, Role: FOLLOWER
I20260811 05:13:18.517647 10004 consensus_queue.cc:260] T e7f858acf89e4211a1fca7ba3bb3fd32 P a65c75fd6fe5471c87c098ddb89930e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } }
I20260811 05:13:18.518407 10003 tablet_bootstrap.cc:492] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752: Bootstrap starting.
I20260811 05:13:18.519413 10003 tablet_bootstrap.cc:654] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:18.520556 10004 ts_tablet_manager.cc:1428] T e7f858acf89e4211a1fca7ba3bb3fd32 P a65c75fd6fe5471c87c098ddb89930e2: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260811 05:13:18.520766 10003 tablet_bootstrap.cc:492] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752: No bootstrap required, opened a new log
I20260811 05:13:18.520859 10003 ts_tablet_manager.cc:1397] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 05:13:18.521274 10003 raft_consensus.cc:348] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } }
I20260811 05:13:18.521379 10003 raft_consensus.cc:374] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:18.521409 10003 raft_consensus.cc:729] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4665ad87d3a47dcaabe060612421752, State: Initialized, Role: FOLLOWER
I20260811 05:13:18.521535 10003 consensus_queue.cc:260] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } }
I20260811 05:13:18.521813 10003 ts_tablet_manager.cc:1428] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 05:13:18.535938 10013 raft_consensus.cc:482] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:18.536053 10013 raft_consensus.cc:504] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } }
I20260811 05:13:18.537292 10013 leader_election.cc:288] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46601d56536e4c439c93bad01b8929cf (127.7.67.129:39569), a65c75fd6fe5471c87c098ddb89930e2 (127.7.67.131:43547)
I20260811 05:13:18.542225 9541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.537536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:43547 (local address 127.7.67.130:49147)
0811 05:13:18.537735 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:18.537753 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:18.537873 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:18.538206 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:18.538210 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:18.538230 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:18.538385 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.538392 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.539429 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.539432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.540841 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.540851 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.541807 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.541811 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.541920 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.541947 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 05:13:18.542023 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 05:13:18.542091 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":60}
I20260811 05:13:18.542520 10000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.537602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:56213 (local address 127.7.67.129:39569)
0811 05:13:18.537721 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:18.537726 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:18.538161 (+ 435us) server_negotiation.cc:407] Connection header received
0811 05:13:18.538388 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:18.538391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:18.538453 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:18.538541 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:18.538883 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.539489 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.541154 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.541813 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.541931 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.541992 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:18.542331 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 05:13:18.542408 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 05:13:18.542549 10052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.537536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:39569 (local address 127.7.67.130:56213)
0811 05:13:18.538068 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:18.538082 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:18.538227 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:18.538639 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:18.538642 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:18.538657 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:18.538765 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.538769 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.539615 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.539619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.540999 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.541011 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.542097 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.542100 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.542187 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.542207 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 05:13:18.542310 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 05:13:18.542382 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"mutex_wait_us":231,"thread_start_us":110,"threads_started":1}
I20260811 05:13:18.543082 9996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.537724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:49147 (local address 127.7.67.131:43547)
0811 05:13:18.537841 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:18.537846 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:18.537860 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:13:18.538035 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:18.538044 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:18.538087 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:18.538195 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:18.538518 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.539305 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.540976 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.541653 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.541787 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.541886 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:18.542890 (+ 1004us) server_negotiation.cc:299] Negotiation successful
0811 05:13:18.542971 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 05:13:18.543251 9414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7f858acf89e4211a1fca7ba3bb3fd32" candidate_uuid: "f4665ad87d3a47dcaabe060612421752" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46601d56536e4c439c93bad01b8929cf" is_pre_election: true
I20260811 05:13:18.543529 9414 raft_consensus.cc:2454] T e7f858acf89e4211a1fca7ba3bb3fd32 P 46601d56536e4c439c93bad01b8929cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4665ad87d3a47dcaabe060612421752 in term 0.
I20260811 05:13:18.544015 9531 leader_election.cc:302] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46601d56536e4c439c93bad01b8929cf, f4665ad87d3a47dcaabe060612421752; no voters:
I20260811 05:13:18.544094 9879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7f858acf89e4211a1fca7ba3bb3fd32" candidate_uuid: "f4665ad87d3a47dcaabe060612421752" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a65c75fd6fe5471c87c098ddb89930e2" is_pre_election: true
I20260811 05:13:18.544278 10013 raft_consensus.cc:2781] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:18.544273 9879 raft_consensus.cc:2454] T e7f858acf89e4211a1fca7ba3bb3fd32 P a65c75fd6fe5471c87c098ddb89930e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4665ad87d3a47dcaabe060612421752 in term 0.
I20260811 05:13:18.544334 10013 raft_consensus.cc:482] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:18.544375 10013 raft_consensus.cc:3037] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:18.545434 10013 raft_consensus.cc:504] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } }
I20260811 05:13:18.545775 10013 leader_election.cc:288] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752 [CANDIDATE]: Term 1 election: Requested vote from peers 46601d56536e4c439c93bad01b8929cf (127.7.67.129:39569), a65c75fd6fe5471c87c098ddb89930e2 (127.7.67.131:43547)
I20260811 05:13:18.546067 9414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7f858acf89e4211a1fca7ba3bb3fd32" candidate_uuid: "f4665ad87d3a47dcaabe060612421752" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46601d56536e4c439c93bad01b8929cf"
I20260811 05:13:18.546190 9414 raft_consensus.cc:3037] T e7f858acf89e4211a1fca7ba3bb3fd32 P 46601d56536e4c439c93bad01b8929cf [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:18.546162 9879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e7f858acf89e4211a1fca7ba3bb3fd32" candidate_uuid: "f4665ad87d3a47dcaabe060612421752" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a65c75fd6fe5471c87c098ddb89930e2"
I20260811 05:13:18.546290 9879 raft_consensus.cc:3037] T e7f858acf89e4211a1fca7ba3bb3fd32 P a65c75fd6fe5471c87c098ddb89930e2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:18.547370 9414 raft_consensus.cc:2454] T e7f858acf89e4211a1fca7ba3bb3fd32 P 46601d56536e4c439c93bad01b8929cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4665ad87d3a47dcaabe060612421752 in term 1.
I20260811 05:13:18.547593 9879 raft_consensus.cc:2454] T e7f858acf89e4211a1fca7ba3bb3fd32 P a65c75fd6fe5471c87c098ddb89930e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4665ad87d3a47dcaabe060612421752 in term 1.
I20260811 05:13:18.547717 9531 leader_election.cc:302] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46601d56536e4c439c93bad01b8929cf, f4665ad87d3a47dcaabe060612421752; no voters:
I20260811 05:13:18.547912 10013 raft_consensus.cc:2781] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:18.548103 10013 raft_consensus.cc:686] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752 [term 1 LEADER]: Becoming Leader. State: Replica: f4665ad87d3a47dcaabe060612421752, State: Running, Role: LEADER
I20260811 05:13:18.548341 10013 consensus_queue.cc:237] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } }
I20260811 05:13:18.550817 9167 catalog_manager.cc:5495] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4665ad87d3a47dcaabe060612421752 (127.7.67.130). New cstate: current_term: 1 leader_uuid: "f4665ad87d3a47dcaabe060612421752" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } health_report { overall_health: UNKNOWN } } }
W20260811 05:13:18.576129 9967 tablet.cc:2369] T e7f858acf89e4211a1fca7ba3bb3fd32 P a65c75fd6fe5471c87c098ddb89930e2: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 05:13:18 dist-test-slave-04bb krb5kdc[9097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425197, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:18.586858 10067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.578278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:41969 (local address 127.0.0.1:55494)
0811 05:13:18.578576 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:18.578587 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:18.578721 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:18.578988 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:18.578991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:18.579413 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:18.579509 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.579517 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.580301 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.580304 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.580936 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.580946 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.581381 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.581385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.581468 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.582083 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:18.582095 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:18.583485 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:18.585249 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:18.585261 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:18.585268 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:18.585496 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:18.585781 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:18.585785 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:18.585787 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:18.585872 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:18.586249 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:18.586256 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:18.586372 (+ 116us) client_negotiation.cc:769] Sending connection context
0811 05:13:18.586490 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 05:13:18.586646 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":111,"threads_started":1}
I20260811 05:13:18.586858 9244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.578353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55494 (local address 127.7.67.190:41969)
0811 05:13:18.578483 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:18.578490 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:18.578665 (+ 175us) server_negotiation.cc:407] Connection header received
0811 05:13:18.578826 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:18.578830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:18.578878 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:18.578965 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:18.579635 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.580182 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.581062 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.581275 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.581380 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.583625 (+ 2245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:18.583629 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:18.583631 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:18.583652 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:18.585123 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:18.585600 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:18.585604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:18.585606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:18.585684 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:18.585961 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:18.585966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:18.585968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:18.586141 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:18.586256 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:18.586538 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 05:13:18.586702 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":43}
I20260811 05:13:18.592056 9167 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:55494:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 05:13:18.592887 9167 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
W20260811 05:13:18.595419 9717 tablet.cc:2369] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:13:18.598274 9388 tablet_service.cc:1467] Processing CreateTablet for tablet ee8cb715460f422abec6a244552badfd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f02d4d5243b84ce2bebbf188cc161edf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:13:18.598402 9609 tablet_service.cc:1467] Processing CreateTablet for tablet ee8cb715460f422abec6a244552badfd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f02d4d5243b84ce2bebbf188cc161edf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:13:18.598793 9609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee8cb715460f422abec6a244552badfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:18.598793 9388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee8cb715460f422abec6a244552badfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:18.601413 10003 tablet_bootstrap.cc:492] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752: Bootstrap starting.
I20260811 05:13:18.601984 9851 tablet_service.cc:1467] Processing CreateTablet for tablet ee8cb715460f422abec6a244552badfd (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f02d4d5243b84ce2bebbf188cc161edf]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:13:18.602351 9851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee8cb715460f422abec6a244552badfd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:18.602411 10003 tablet_bootstrap.cc:654] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:18.605351 10003 tablet_bootstrap.cc:492] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752: No bootstrap required, opened a new log
I20260811 05:13:18.605377 10004 tablet_bootstrap.cc:492] T ee8cb715460f422abec6a244552badfd P a65c75fd6fe5471c87c098ddb89930e2: Bootstrap starting.
I20260811 05:13:18.605430 10003 ts_tablet_manager.cc:1397] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260811 05:13:18.606130 10003 raft_consensus.cc:348] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } }
I20260811 05:13:18.606244 10003 raft_consensus.cc:374] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:18.606272 10003 raft_consensus.cc:729] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4665ad87d3a47dcaabe060612421752, State: Initialized, Role: FOLLOWER
I20260811 05:13:18.606377 10004 tablet_bootstrap.cc:654] T ee8cb715460f422abec6a244552badfd P a65c75fd6fe5471c87c098ddb89930e2: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:18.606385 10003 consensus_queue.cc:260] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } }
I20260811 05:13:18.606635 10003 ts_tablet_manager.cc:1428] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 05:13:18.607398 10013 tablet_replica.cc:439] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } } }
I20260811 05:13:18.607525 10013 tablet_replica.cc:442] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:18.607613 10004 tablet_bootstrap.cc:492] T ee8cb715460f422abec6a244552badfd P a65c75fd6fe5471c87c098ddb89930e2: No bootstrap required, opened a new log
I20260811 05:13:18.607692 10004 ts_tablet_manager.cc:1397] T ee8cb715460f422abec6a244552badfd P a65c75fd6fe5471c87c098ddb89930e2: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:13:18.608141 10004 raft_consensus.cc:348] T ee8cb715460f422abec6a244552badfd P a65c75fd6fe5471c87c098ddb89930e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } }
I20260811 05:13:18.608258 10004 raft_consensus.cc:374] T ee8cb715460f422abec6a244552badfd P a65c75fd6fe5471c87c098ddb89930e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:18.608291 10004 raft_consensus.cc:729] T ee8cb715460f422abec6a244552badfd P a65c75fd6fe5471c87c098ddb89930e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a65c75fd6fe5471c87c098ddb89930e2, State: Initialized, Role: FOLLOWER
I20260811 05:13:18.608407 10004 consensus_queue.cc:260] T ee8cb715460f422abec6a244552badfd P a65c75fd6fe5471c87c098ddb89930e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } }
I20260811 05:13:18.608482 10001 tablet_bootstrap.cc:492] T ee8cb715460f422abec6a244552badfd P 46601d56536e4c439c93bad01b8929cf: Bootstrap starting.
I20260811 05:13:18.608681 10004 ts_tablet_manager.cc:1428] T ee8cb715460f422abec6a244552badfd P a65c75fd6fe5471c87c098ddb89930e2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:18.609532 10001 tablet_bootstrap.cc:654] T ee8cb715460f422abec6a244552badfd P 46601d56536e4c439c93bad01b8929cf: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:18.609815 10018 tablet_replica.cc:439] T ee8cb715460f422abec6a244552badfd P a65c75fd6fe5471c87c098ddb89930e2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } } }
I20260811 05:13:18.609977 10018 tablet_replica.cc:442] T ee8cb715460f422abec6a244552badfd P a65c75fd6fe5471c87c098ddb89930e2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:18.611442 10001 tablet_bootstrap.cc:492] T ee8cb715460f422abec6a244552badfd P 46601d56536e4c439c93bad01b8929cf: No bootstrap required, opened a new log
I20260811 05:13:18.611539 10001 ts_tablet_manager.cc:1397] T ee8cb715460f422abec6a244552badfd P 46601d56536e4c439c93bad01b8929cf: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 05:13:18.611977 10001 raft_consensus.cc:348] T ee8cb715460f422abec6a244552badfd P 46601d56536e4c439c93bad01b8929cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } }
I20260811 05:13:18.612092 10001 raft_consensus.cc:374] T ee8cb715460f422abec6a244552badfd P 46601d56536e4c439c93bad01b8929cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:18.612126 10001 raft_consensus.cc:729] T ee8cb715460f422abec6a244552badfd P 46601d56536e4c439c93bad01b8929cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46601d56536e4c439c93bad01b8929cf, State: Initialized, Role: FOLLOWER
I20260811 05:13:18.612262 10001 consensus_queue.cc:260] T ee8cb715460f422abec6a244552badfd P 46601d56536e4c439c93bad01b8929cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } }
I20260811 05:13:18.612593 10001 ts_tablet_manager.cc:1428] T ee8cb715460f422abec6a244552badfd P 46601d56536e4c439c93bad01b8929cf: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 05:13:18.613091 10035 tablet_replica.cc:439] T ee8cb715460f422abec6a244552badfd P 46601d56536e4c439c93bad01b8929cf: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } } }
I20260811 05:13:18.613288 10035 tablet_replica.cc:442] T ee8cb715460f422abec6a244552badfd P 46601d56536e4c439c93bad01b8929cf: This TxnStatusTablet replica's current role is: FOLLOWER
W20260811 05:13:18.621304 9489 tablet.cc:2369] T e7f858acf89e4211a1fca7ba3bb3fd32 P 46601d56536e4c439c93bad01b8929cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:13:18.705853 10013 raft_consensus.cc:482] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:18.705993 10013 raft_consensus.cc:504] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } }
I20260811 05:13:18.706473 10013 leader_election.cc:288] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 46601d56536e4c439c93bad01b8929cf (127.7.67.129:39569), a65c75fd6fe5471c87c098ddb89930e2 (127.7.67.131:43547)
I20260811 05:13:18.706969 9414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee8cb715460f422abec6a244552badfd" candidate_uuid: "f4665ad87d3a47dcaabe060612421752" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46601d56536e4c439c93bad01b8929cf" is_pre_election: true
I20260811 05:13:18.707064 9879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee8cb715460f422abec6a244552badfd" candidate_uuid: "f4665ad87d3a47dcaabe060612421752" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a65c75fd6fe5471c87c098ddb89930e2" is_pre_election: true
I20260811 05:13:18.707160 9414 raft_consensus.cc:2454] T ee8cb715460f422abec6a244552badfd P 46601d56536e4c439c93bad01b8929cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4665ad87d3a47dcaabe060612421752 in term 0.
I20260811 05:13:18.707293 9879 raft_consensus.cc:2454] T ee8cb715460f422abec6a244552badfd P a65c75fd6fe5471c87c098ddb89930e2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4665ad87d3a47dcaabe060612421752 in term 0.
I20260811 05:13:18.707508 9531 leader_election.cc:302] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46601d56536e4c439c93bad01b8929cf, f4665ad87d3a47dcaabe060612421752; no voters:
I20260811 05:13:18.707713 10013 raft_consensus.cc:2781] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:18.707785 10013 raft_consensus.cc:482] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:18.707841 10013 raft_consensus.cc:3037] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:18.708909 10013 raft_consensus.cc:504] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } }
I20260811 05:13:18.709282 10013 leader_election.cc:288] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752 [CANDIDATE]: Term 1 election: Requested vote from peers 46601d56536e4c439c93bad01b8929cf (127.7.67.129:39569), a65c75fd6fe5471c87c098ddb89930e2 (127.7.67.131:43547)
I20260811 05:13:18.709613 9414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee8cb715460f422abec6a244552badfd" candidate_uuid: "f4665ad87d3a47dcaabe060612421752" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "46601d56536e4c439c93bad01b8929cf"
I20260811 05:13:18.709694 9879 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee8cb715460f422abec6a244552badfd" candidate_uuid: "f4665ad87d3a47dcaabe060612421752" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a65c75fd6fe5471c87c098ddb89930e2"
I20260811 05:13:18.709779 9414 raft_consensus.cc:3037] T ee8cb715460f422abec6a244552badfd P 46601d56536e4c439c93bad01b8929cf [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:18.709805 9879 raft_consensus.cc:3037] T ee8cb715460f422abec6a244552badfd P a65c75fd6fe5471c87c098ddb89930e2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:18.710734 9414 raft_consensus.cc:2454] T ee8cb715460f422abec6a244552badfd P 46601d56536e4c439c93bad01b8929cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4665ad87d3a47dcaabe060612421752 in term 1.
I20260811 05:13:18.710742 9879 raft_consensus.cc:2454] T ee8cb715460f422abec6a244552badfd P a65c75fd6fe5471c87c098ddb89930e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4665ad87d3a47dcaabe060612421752 in term 1.
I20260811 05:13:18.711053 9531 leader_election.cc:302] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 46601d56536e4c439c93bad01b8929cf, f4665ad87d3a47dcaabe060612421752; no voters:
I20260811 05:13:18.711287 10013 raft_consensus.cc:2781] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:18.711380 10013 raft_consensus.cc:686] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752 [term 1 LEADER]: Becoming Leader. State: Replica: f4665ad87d3a47dcaabe060612421752, State: Running, Role: LEADER
I20260811 05:13:18.711514 10013 consensus_queue.cc:237] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } }
I20260811 05:13:18.712225 10053 tablet_replica.cc:439] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752: TxnStatusTablet state changed. Reason: New leader f4665ad87d3a47dcaabe060612421752. Latest consensus state: current_term: 1 leader_uuid: "f4665ad87d3a47dcaabe060612421752" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } } }
I20260811 05:13:18.712340 10053 tablet_replica.cc:442] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:13:18.712706 10075 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 05:13:18.712939 10075 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 05:13:18.713415 9169 catalog_manager.cc:5495] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4665ad87d3a47dcaabe060612421752 (127.7.67.130). New cstate: current_term: 1 leader_uuid: "f4665ad87d3a47dcaabe060612421752" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } health_report { overall_health: UNKNOWN } } }
I20260811 05:13:18.742749 9995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.738541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57244 (local address 127.7.67.130:44051)
0811 05:13:18.738667 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:18.738671 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:18.738720 (+ 49us) server_negotiation.cc:407] Connection header received
0811 05:13:18.738922 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:18.738924 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:18.738977 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:18.739061 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:18.739338 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.739826 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.741136 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.741402 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.741569 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.741898 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:18.742364 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:18.742455 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:18.742577 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 05:13:18.742627 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 05:13:18.742777 10067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.738435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:44051 (local address 127.0.0.1:57244)
0811 05:13:18.738662 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:18.738676 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:18.738786 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:18.739099 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:18.739102 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:18.739110 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:13:18.739231 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.739237 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.739928 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.739932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.740960 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.740972 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.741580 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.741584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.741686 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.741744 (+ 58us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:13:18.742461 (+ 717us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:13:18.742477 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 05:13:18.742551 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 05:13:18.742635 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128}
I20260811 05:13:18.746109 9879 raft_consensus.cc:1261] T ee8cb715460f422abec6a244552badfd P a65c75fd6fe5471c87c098ddb89930e2 [term 1 FOLLOWER]: Refusing update from remote peer f4665ad87d3a47dcaabe060612421752: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:18.746212 9414 raft_consensus.cc:1261] T ee8cb715460f422abec6a244552badfd P 46601d56536e4c439c93bad01b8929cf [term 1 FOLLOWER]: Refusing update from remote peer f4665ad87d3a47dcaabe060612421752: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:18.746650 10013 consensus_queue.cc:1035] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:18.746816 10053 consensus_queue.cc:1035] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:18.749055 10035 tablet_replica.cc:439] T ee8cb715460f422abec6a244552badfd P 46601d56536e4c439c93bad01b8929cf: TxnStatusTablet state changed. Reason: New leader f4665ad87d3a47dcaabe060612421752. Latest consensus state: current_term: 1 leader_uuid: "f4665ad87d3a47dcaabe060612421752" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } } }
I20260811 05:13:18.749181 10035 tablet_replica.cc:442] T ee8cb715460f422abec6a244552badfd P 46601d56536e4c439c93bad01b8929cf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:18.751325 10018 tablet_replica.cc:439] T ee8cb715460f422abec6a244552badfd P a65c75fd6fe5471c87c098ddb89930e2: TxnStatusTablet state changed. Reason: New leader f4665ad87d3a47dcaabe060612421752. Latest consensus state: current_term: 1 leader_uuid: "f4665ad87d3a47dcaabe060612421752" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } } }
I20260811 05:13:18.751418 10018 tablet_replica.cc:442] T ee8cb715460f422abec6a244552badfd P a65c75fd6fe5471c87c098ddb89930e2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:18.752583 10053 tablet_replica.cc:439] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f4665ad87d3a47dcaabe060612421752" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } } }
I20260811 05:13:18.752619 10013 tablet_replica.cc:439] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f4665ad87d3a47dcaabe060612421752" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } } }
I20260811 05:13:18.752689 10053 tablet_replica.cc:442] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:13:18.752715 10013 tablet_replica.cc:442] T ee8cb715460f422abec6a244552badfd P f4665ad87d3a47dcaabe060612421752: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:13:18.753261 10030 mvcc.cc:205] Tried to move back new op lower bound from 7317197614058713088 to 7317197613923889152. Current Snapshot: MvccSnapshot[applied={T|T < 7317197614058713088 or (T in {7317197614058713088})}]
I20260811 05:13:18.754135 10026 mvcc.cc:205] Tried to move back new op lower bound from 7317197614058713088 to 7317197613923889152. Current Snapshot: MvccSnapshot[applied={T|T < 7317197614058713088}]
I20260811 05:13:18.754237 10018 tablet_replica.cc:439] T ee8cb715460f422abec6a244552badfd P a65c75fd6fe5471c87c098ddb89930e2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f4665ad87d3a47dcaabe060612421752" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } } }
I20260811 05:13:18.754339 10018 tablet_replica.cc:442] T ee8cb715460f422abec6a244552badfd P a65c75fd6fe5471c87c098ddb89930e2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:18.754323 10035 tablet_replica.cc:439] T ee8cb715460f422abec6a244552badfd P 46601d56536e4c439c93bad01b8929cf: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f4665ad87d3a47dcaabe060612421752" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 } } peers { permanent_uuid: "f4665ad87d3a47dcaabe060612421752" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44051 } } peers { permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 } } }
I20260811 05:13:18.754416 10035 tablet_replica.cc:442] T ee8cb715460f422abec6a244552badfd P 46601d56536e4c439c93bad01b8929cf: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:18.761474 9995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.758323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57252 (local address 127.7.67.130:44051)
0811 05:13:18.758436 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:18.758441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:18.758454 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:13:18.758552 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:18.758556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:18.758596 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:18.758685 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:18.758946 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.759444 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.760367 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.760665 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.760792 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.761045 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:18.761120 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:18.761201 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:18.761304 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 05:13:18.761349 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 05:13:18.773183 9995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.769875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:60437 (local address 127.7.67.130:44051)
0811 05:13:18.770000 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:18.770004 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:18.770125 (+ 121us) server_negotiation.cc:407] Connection header received
0811 05:13:18.770343 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:18.770347 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:18.770386 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:18.770469 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:18.770703 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.771181 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.772028 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.772240 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.772350 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.772698 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:18.772772 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:18.772848 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:18.772938 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 05:13:18.773032 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":68}
I20260811 05:13:18.773192 10086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.769724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:44051 (local address 127.7.67.130:60437)
0811 05:13:18.770032 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:18.770046 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:18.770181 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:18.770478 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:18.770481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:18.770487 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:13:18.770598 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.770608 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.771291 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.771294 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.771911 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:18.771919 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.772365 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.772368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:18.772486 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.772572 (+ 86us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:13:18.772863 (+ 291us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:13:18.772870 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 05:13:18.772929 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 05:13:18.773048 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":41,"thread_start_us":95,"threads_started":1}
I20260811 05:13:18.779546 9879 raft_consensus.cc:1261] T e7f858acf89e4211a1fca7ba3bb3fd32 P a65c75fd6fe5471c87c098ddb89930e2 [term 1 FOLLOWER]: Refusing update from remote peer f4665ad87d3a47dcaabe060612421752: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:18.779907 9414 raft_consensus.cc:1261] T e7f858acf89e4211a1fca7ba3bb3fd32 P 46601d56536e4c439c93bad01b8929cf [term 1 FOLLOWER]: Refusing update from remote peer f4665ad87d3a47dcaabe060612421752: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:18.780054 10082 consensus_queue.cc:1035] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a65c75fd6fe5471c87c098ddb89930e2" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43547 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:18.780293 10087 consensus_queue.cc:1035] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752 [LEADER]: Connected to new peer: Peer: permanent_uuid: "46601d56536e4c439c93bad01b8929cf" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 39569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:18.823120 9169 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:18.823267 9169 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:18.825259 9169 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 40a9f150d52c4cca83a073997534cd55: Sending DeleteTablet for 3 replicas of tablet e7f858acf89e4211a1fca7ba3bb3fd32
I20260811 05:13:18.825860 9388 tablet_service.cc:1514] Processing DeleteTablet for tablet e7f858acf89e4211a1fca7ba3bb3fd32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:18 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:58452
I20260811 05:13:18.825904 9851 tablet_service.cc:1514] Processing DeleteTablet for tablet e7f858acf89e4211a1fca7ba3bb3fd32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:18 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:36788
I20260811 05:13:18.826041 10044 tablet_replica.cc:330] T e7f858acf89e4211a1fca7ba3bb3fd32 P 46601d56536e4c439c93bad01b8929cf: stopping tablet replica
I20260811 05:13:18.826085 10041 tablet_replica.cc:330] T e7f858acf89e4211a1fca7ba3bb3fd32 P a65c75fd6fe5471c87c098ddb89930e2: stopping tablet replica
I20260811 05:13:18.826166 10044 raft_consensus.cc:2229] T e7f858acf89e4211a1fca7ba3bb3fd32 P 46601d56536e4c439c93bad01b8929cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:18.826189 10041 raft_consensus.cc:2229] T e7f858acf89e4211a1fca7ba3bb3fd32 P a65c75fd6fe5471c87c098ddb89930e2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:18.826288 10044 raft_consensus.cc:2258] T e7f858acf89e4211a1fca7ba3bb3fd32 P 46601d56536e4c439c93bad01b8929cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:18.826288 10041 raft_consensus.cc:2258] T e7f858acf89e4211a1fca7ba3bb3fd32 P a65c75fd6fe5471c87c098ddb89930e2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:18.826746 9609 tablet_service.cc:1514] Processing DeleteTablet for tablet e7f858acf89e4211a1fca7ba3bb3fd32 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:18 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:57238
I20260811 05:13:18.826905 10040 tablet_replica.cc:330] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752: stopping tablet replica
I20260811 05:13:18.827004 10040 raft_consensus.cc:2229] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:13:18.827273 10040 raft_consensus.cc:2258] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:18.827728 10041 ts_tablet_manager.cc:1905] T e7f858acf89e4211a1fca7ba3bb3fd32 P a65c75fd6fe5471c87c098ddb89930e2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:18.828111 10040 ts_tablet_manager.cc:1905] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:18.828331 10044 ts_tablet_manager.cc:1905] T e7f858acf89e4211a1fca7ba3bb3fd32 P 46601d56536e4c439c93bad01b8929cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:18.830709 10041 ts_tablet_manager.cc:1918] T e7f858acf89e4211a1fca7ba3bb3fd32 P a65c75fd6fe5471c87c098ddb89930e2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:13:18.830792 10041 log.cc:1196] T e7f858acf89e4211a1fca7ba3bb3fd32 P a65c75fd6fe5471c87c098ddb89930e2: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/e7f858acf89e4211a1fca7ba3bb3fd32
I20260811 05:13:18.831077 10041 ts_tablet_manager.cc:1939] T e7f858acf89e4211a1fca7ba3bb3fd32 P a65c75fd6fe5471c87c098ddb89930e2: Deleting consensus metadata
I20260811 05:13:18.831140 10040 ts_tablet_manager.cc:1918] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:13:18.831213 10040 log.cc:1196] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/e7f858acf89e4211a1fca7ba3bb3fd32
I20260811 05:13:18.831537 10040 ts_tablet_manager.cc:1939] T e7f858acf89e4211a1fca7ba3bb3fd32 P f4665ad87d3a47dcaabe060612421752: Deleting consensus metadata
I20260811 05:13:18.831537 10044 ts_tablet_manager.cc:1918] T e7f858acf89e4211a1fca7ba3bb3fd32 P 46601d56536e4c439c93bad01b8929cf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:13:18.831710 10044 log.cc:1196] T e7f858acf89e4211a1fca7ba3bb3fd32 P 46601d56536e4c439c93bad01b8929cf: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786425190664461-7438-0/minicluster-data/ts-0/wal/wals/e7f858acf89e4211a1fca7ba3bb3fd32
I20260811 05:13:18.831851 9145 catalog_manager.cc:4841] TS a65c75fd6fe5471c87c098ddb89930e2 (127.7.67.131:43547): tablet e7f858acf89e4211a1fca7ba3bb3fd32 (table test-table [id=4a6052d07e86429f8d34c786472ebc76]) successfully deleted
I20260811 05:13:18.832320 10044 ts_tablet_manager.cc:1939] T e7f858acf89e4211a1fca7ba3bb3fd32 P 46601d56536e4c439c93bad01b8929cf: Deleting consensus metadata
I20260811 05:13:18.832289 9143 catalog_manager.cc:4841] TS f4665ad87d3a47dcaabe060612421752 (127.7.67.130:44051): tablet e7f858acf89e4211a1fca7ba3bb3fd32 (table test-table [id=4a6052d07e86429f8d34c786472ebc76]) successfully deleted
I20260811 05:13:18.833173 9143 catalog_manager.cc:4841] TS 46601d56536e4c439c93bad01b8929cf (127.7.67.129:39569): tablet e7f858acf89e4211a1fca7ba3bb3fd32 (table test-table [id=4a6052d07e86429f8d34c786472ebc76]) successfully deleted
Aug 11 05:13:18 dist-test-slave-04bb krb5kdc[9097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425198, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.67.129@KRBTEST.COM
I20260811 05:13:18.847663 10000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.827087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58464 (local address 127.7.67.129:39569)
0811 05:13:18.827470 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:18.827475 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:18.832390 (+ 4915us) server_negotiation.cc:407] Connection header received
0811 05:13:18.832633 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:18.832636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:18.832686 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:18.832778 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:18.833672 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.834183 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.835108 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.835352 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.835463 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.844021 (+ 8558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:18.844040 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:18.844052 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:18.844076 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:18.845869 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:18.846315 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:18.846321 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:18.846322 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:18.846401 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:18.846687 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:18.846691 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:18.846693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:18.846953 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:18.847166 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:18.847374 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 05:13:18.847502 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299}
W20260811 05:13:18.848152 9478 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58464
I20260811 05:13:18.861927 9244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:18.850081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55510 (local address 127.7.67.190:41969)
0811 05:13:18.850229 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:18.850233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:18.850251 (+ 18us) server_negotiation.cc:407] Connection header received
0811 05:13:18.850409 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:18.850413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:18.850470 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:18.850557 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:18.851495 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.852351 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.853647 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:18.853999 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:18.854120 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:18.855952 (+ 1832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:18.855957 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:18.855960 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:18.855991 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:18.858099 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:18.858725 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:18.858730 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:18.858732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:18.858833 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:18.860840 (+ 2007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:18.860846 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:18.860850 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:18.861096 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:18.861213 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:18.861549 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 05:13:18.861685 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 05:13:18.863083 9169 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55510
I20260811 05:13:18.864696 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 9279
I20260811 05:13:18.892879 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 9499
I20260811 05:13:18.910918 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 9729
I20260811 05:13:18.925990 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 9116
2026-08-11T05:13:18Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2954 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:13:18 dist-test-slave-04bb krb5kdc[10103](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:13:18 dist-test-slave-04bb krb5kdc[10103](info): set up 2 sockets
Aug 11 05:13:18 dist-test-slave-04bb krb5kdc[10103](info): commencing operation
krb5kdc: starting...
W20260811 05:13:20.256438 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.299s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:13:20 dist-test-slave-04bb krb5kdc[10103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:13:20.391885 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:35425
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:35049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:35425
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:20.505208 10124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:20.505501 10124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:20.505578 10124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:20.509521 10124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:20.509609 10124 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 05:13:20.509641 10124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:20.509677 10124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:20.509711 10124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:20.509740 10124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:20.514860 10124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:35049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:35425
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:35425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:20.515915 10124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:20.517020 10124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:20.525701 10130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:20.525866 10124 server_base.cc:1034] running on GCE node
W20260811 05:13:20.525701 10131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:20.525931 10134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:20.526624 10124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:20.527748 10124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:20.528946 10124 hybrid_clock.cc:648] HybridClock initialized: now 1786425200528915 us; error 40 us; skew 500 ppm
Aug 11 05:13:20 dist-test-slave-04bb krb5kdc[10103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425200, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:20.535593 10124 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:13:20.536572 10124 webserver.cc:460] Webserver started at http://127.7.67.190:43497/ using document root <none> and password file <none>
I20260811 05:13:20.537169 10124 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:20.537294 10124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:20.537534 10124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:20.539608 10124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "688908b3559d42219236a70e7cb58115"
format_stamp: "Formatted at 2026-08-11 05:13:20 on dist-test-slave-04bb"
server_key: "b155c24d1df11fd7b70200e27f5ab1b0"
server_key_iv: "dd30008f46187ebf21882cc83df464d6"
server_key_version: "encryptionkey@0"
I20260811 05:13:20.540189 10124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "688908b3559d42219236a70e7cb58115"
format_stamp: "Formatted at 2026-08-11 05:13:20 on dist-test-slave-04bb"
server_key: "b155c24d1df11fd7b70200e27f5ab1b0"
server_key_iv: "dd30008f46187ebf21882cc83df464d6"
server_key_version: "encryptionkey@0"
I20260811 05:13:20.543917 10124 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 05:13:20.546437 10147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:20.547688 10124 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 05:13:20.547856 10124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "688908b3559d42219236a70e7cb58115"
format_stamp: "Formatted at 2026-08-11 05:13:20 on dist-test-slave-04bb"
server_key: "b155c24d1df11fd7b70200e27f5ab1b0"
server_key_iv: "dd30008f46187ebf21882cc83df464d6"
server_key_version: "encryptionkey@0"
I20260811 05:13:20.548004 10124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:20.568902 10124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:20.571892 10124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:20.572108 10124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:20.579999 10124 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:35425
I20260811 05:13:20.580029 10235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:35425 every 8 connection(s)
I20260811 05:13:20.580958 10124 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:20.583705 10237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:20.588871 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 10124
I20260811 05:13:20.588990 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:13:20.589314 7438 external_mini_cluster.cc:1428] Setting key 9b7fe86737db35fd9d282ac855709b9a
I20260811 05:13:20.591369 10237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115: Bootstrap starting.
I20260811 05:13:20.593688 10237 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:20.594453 10237 log.cc:824] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:20.596782 10237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115: No bootstrap required, opened a new log
Aug 11 05:13:20 dist-test-slave-04bb krb5kdc[10103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:20.601263 10237 raft_consensus.cc:348] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "688908b3559d42219236a70e7cb58115" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 35425 } }
I20260811 05:13:20.601486 10237 raft_consensus.cc:374] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:20.601522 10237 raft_consensus.cc:729] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 688908b3559d42219236a70e7cb58115, State: Initialized, Role: FOLLOWER
I20260811 05:13:20.602087 10237 consensus_queue.cc:260] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "688908b3559d42219236a70e7cb58115" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 35425 } }
I20260811 05:13:20.602278 10237 raft_consensus.cc:388] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:20.602365 10237 raft_consensus.cc:482] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:20.602468 10237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:20.603577 10237 raft_consensus.cc:504] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "688908b3559d42219236a70e7cb58115" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 35425 } }
I20260811 05:13:20.603965 10237 leader_election.cc:302] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 688908b3559d42219236a70e7cb58115; no voters:
I20260811 05:13:20.604338 10237 leader_election.cc:288] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:20.604447 10243 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:20.604775 10243 raft_consensus.cc:686] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115 [term 1 LEADER]: Becoming Leader. State: Replica: 688908b3559d42219236a70e7cb58115, State: Running, Role: LEADER
I20260811 05:13:20.605084 10243 consensus_queue.cc:237] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "688908b3559d42219236a70e7cb58115" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 35425 } }
I20260811 05:13:20.605283 10240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:20.590909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50320 (local address 127.7.67.190:35425)
0811 05:13:20.591401 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:20.591416 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:20.591485 (+ 69us) server_negotiation.cc:407] Connection header received
0811 05:13:20.592142 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:20.592161 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:20.592493 (+ 332us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:20.592902 (+ 409us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:20.593749 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.594847 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.596162 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.596909 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.597074 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:20.599832 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:20.599837 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:20.599851 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:20.599877 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:20.601851 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:20.602354 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:20.602360 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:20.602367 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:20.602538 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:20.602900 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:20.602903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:20.602904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:20.603293 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:20.603439 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:20.603866 (+ 427us) server_negotiation.cc:299] Negotiation successful
0811 05:13:20.604196 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":139,"threads_started":1}
I20260811 05:13:20.605786 10237 sys_catalog.cc:564] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:20.606271 10243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 688908b3559d42219236a70e7cb58115. Latest consensus state: current_term: 1 leader_uuid: "688908b3559d42219236a70e7cb58115" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "688908b3559d42219236a70e7cb58115" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 35425 } } }
I20260811 05:13:20.606387 10243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:20.607053 10246 sys_catalog.cc:455] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "688908b3559d42219236a70e7cb58115" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "688908b3559d42219236a70e7cb58115" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 35425 } } }
I20260811 05:13:20.607156 10246 sys_catalog.cc:458] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:20.607782 10248 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:20.610780 10248 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:20.616493 10248 catalog_manager.cc:1347] Generated new cluster ID: 3eb72eb1328d4a20a2c0f5d005211d29
I20260811 05:13:20.616597 10248 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:20.627722 10248 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:20.628660 10248 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:20.635231 10248 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115: Generated new TSK 0
I20260811 05:13:20.635849 10248 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
I20260811 05:13:20.669234 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:35425
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:35049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:20.787796 10281 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:20.788154 10281 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:20.788249 10281 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:20.792673 10281 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:20.792727 10281 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:20.792768 10281 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 05:13:20.792783 10281 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:13:20.792863 10281 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:13:20.798177 10281 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:35049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.67.190:35425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10281
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:20.799209 10281 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:20.800233 10281 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:20.810405 10296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:20.810446 10289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:20.810564 10281 server_base.cc:1034] running on GCE node
W20260811 05:13:20.810446 10290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:20.811172 10281 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:20.811810 10281 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:20.813019 10281 hybrid_clock.cc:648] HybridClock initialized: now 1786425200813000 us; error 30 us; skew 500 ppm
Aug 11 05:13:20 dist-test-slave-04bb krb5kdc[10103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425200, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:20.816010 10281 init.cc:377] Logged in from keytab as kudu/127.7.67.129@KRBTEST.COM (short username kudu)
I20260811 05:13:20.816957 10281 webserver.cc:460] Webserver started at http://127.7.67.129:43215/ using document root <none> and password file <none>
I20260811 05:13:20.817533 10281 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:20.817632 10281 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:20.817873 10281 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:20.819907 10281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "220563d55ccc4fac99d9930a81a07316"
format_stamp: "Formatted at 2026-08-11 05:13:20 on dist-test-slave-04bb"
server_key: "3745daf1362f122fc0c48954bebc8434"
server_key_iv: "70bd833e62bba20c24e73f155cc36600"
server_key_version: "encryptionkey@0"
I20260811 05:13:20.820479 10281 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "220563d55ccc4fac99d9930a81a07316"
format_stamp: "Formatted at 2026-08-11 05:13:20 on dist-test-slave-04bb"
server_key: "3745daf1362f122fc0c48954bebc8434"
server_key_iv: "70bd833e62bba20c24e73f155cc36600"
server_key_version: "encryptionkey@0"
I20260811 05:13:20.824414 10281 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 05:13:20.826982 10303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:20.828517 10281 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 05:13:20.828751 10281 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "220563d55ccc4fac99d9930a81a07316"
format_stamp: "Formatted at 2026-08-11 05:13:20 on dist-test-slave-04bb"
server_key: "3745daf1362f122fc0c48954bebc8434"
server_key_iv: "70bd833e62bba20c24e73f155cc36600"
server_key_version: "encryptionkey@0"
I20260811 05:13:20.828878 10281 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:20.846249 10281 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:20.849608 10281 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:20.849786 10281 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:20.851402 10281 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:20.851491 10281 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:20.851562 10281 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:20.851601 10281 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 05:13:20 dist-test-slave-04bb krb5kdc[10103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425200, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:20.863030 10281 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:43129
I20260811 05:13:20.863116 10482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:43129 every 8 connection(s)
I20260811 05:13:20.864048 10281 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:20.866197 10240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:20.852207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:39143 (local address 127.7.67.190:35425)
0811 05:13:20.852387 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:20.852394 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:20.853378 (+ 984us) server_negotiation.cc:407] Connection header received
0811 05:13:20.854455 (+ 1077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:20.854459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:20.854523 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:20.854583 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:20.856479 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.857542 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.858906 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.859300 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.859412 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:20.862294 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:20.862297 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:20.862300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:20.862328 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:20.863974 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:20.864520 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:20.864546 (+ 26us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:20.864551 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:20.864645 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:20.865089 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:20.865093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:20.865095 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:20.865293 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:20.865410 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:20.865896 (+ 486us) server_negotiation.cc:299] Negotiation successful
0811 05:13:20.866021 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 05:13:20.866976 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 10281
I20260811 05:13:20.867046 10316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:20.852712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:35425 (local address 127.7.67.129:39143)
0811 05:13:20.853190 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:20.853241 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:20.854175 (+ 934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:20.855044 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:20.855055 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:20.855555 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:20.856276 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:20.856299 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.857703 (+ 1404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.857707 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:20.858768 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:20.858778 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.859449 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.859453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:20.859649 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:20.860310 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:20.860325 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:20.862119 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:20.864099 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:20.864105 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:20.864120 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:20.864405 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:20.864788 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:20.864793 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:20.864798 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:20.864981 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:20.865408 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:20.865415 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:20.865652 (+ 237us) client_negotiation.cc:769] Sending connection context
0811 05:13:20.865848 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 05:13:20.866154 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":131,"threads_started":1}
I20260811 05:13:20.867110 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:13:20.867429 7438 external_mini_cluster.cc:1428] Setting key 1d6ff0db1c053805eaeea37e9496ae1e
I20260811 05:13:20.868803 10483 heartbeater.cc:347] Connected to a master server at 127.7.67.190:35425
I20260811 05:13:20.868996 10483 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:20.869550 10483 heartbeater.cc:511] Master 127.7.67.190:35425 requested a full tablet report, sending...
I20260811 05:13:20.871253 10171 ts_manager.cc:194] Registered new tserver with Master: 220563d55ccc4fac99d9930a81a07316 (127.7.67.129:43129)
I20260811 05:13:20.872956 10171 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.129@KRBTEST.COM'} at 127.7.67.129:39143
I20260811 05:13:20.878908 10240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:20.871245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:54257 (local address 127.7.67.190:35425)
0811 05:13:20.871375 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:20.871381 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:20.871634 (+ 253us) server_negotiation.cc:407] Connection header received
0811 05:13:20.871735 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:20.871737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:20.871787 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:20.871901 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:20.872619 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.873182 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.874174 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.874408 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.874521 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:20.875687 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:20.875691 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:20.875694 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:20.875718 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:20.877197 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:20.877690 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:20.877693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:20.877695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:20.877776 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:20.878076 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:20.878081 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:20.878083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:20.878263 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:20.878376 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:20.878637 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 05:13:20.878753 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 05:13:20.878928 10493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:20.871150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:35425 (local address 127.7.67.129:54257)
0811 05:13:20.871542 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:20.871557 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:20.871655 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:20.872059 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:20.872062 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:20.872309 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:20.872462 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:20.872470 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.873323 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.873326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:20.874050 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:20.874059 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:20.874529 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:20.874532 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:20.874621 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:20.875157 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:20.875169 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:20.875547 (+ 378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:20.877345 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:20.877348 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:20.877350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:20.877577 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:20.877873 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:20.877876 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:20.877878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:20.877976 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:20.878367 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:20.878370 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:20.878468 (+ 98us) client_negotiation.cc:769] Sending connection context
0811 05:13:20.878581 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 05:13:20.878726 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":155,"threads_started":1}
WARNING: no policy specified for kudu/127.7.67.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
I20260811 05:13:20.898936 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:35425
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:35049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:21.011816 10496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:21.012054 10496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:21.012101 10496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:21.016094 10496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:21.016151 10496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:21.016191 10496 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 05:13:21.016204 10496 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:13:21.016280 10496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:13:21.021412 10496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:35049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.67.190:35425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:21.022459 10496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:21.023519 10496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:21.033231 10504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:21.033381 10507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:21.033427 10496 server_base.cc:1034] running on GCE node
W20260811 05:13:21.033543 10502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:21.033977 10496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:21.034698 10496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:21.035893 10496 hybrid_clock.cc:648] HybridClock initialized: now 1786425201035858 us; error 43 us; skew 500 ppm
Aug 11 05:13:21 dist-test-slave-04bb krb5kdc[10103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425201, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:21.038991 10496 init.cc:377] Logged in from keytab as kudu/127.7.67.130@KRBTEST.COM (short username kudu)
I20260811 05:13:21.039839 10496 webserver.cc:460] Webserver started at http://127.7.67.130:45721/ using document root <none> and password file <none>
I20260811 05:13:21.040376 10496 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:21.040441 10496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:21.040695 10496 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:21.042685 10496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "ade76aa26d02404682b0798c158915aa"
format_stamp: "Formatted at 2026-08-11 05:13:21 on dist-test-slave-04bb"
server_key: "db6317aefa244aee758b02eac598c2d7"
server_key_iv: "7291128d72f0b1e708dcb2bda3921a25"
server_key_version: "encryptionkey@0"
I20260811 05:13:21.043229 10496 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "ade76aa26d02404682b0798c158915aa"
format_stamp: "Formatted at 2026-08-11 05:13:21 on dist-test-slave-04bb"
server_key: "db6317aefa244aee758b02eac598c2d7"
server_key_iv: "7291128d72f0b1e708dcb2bda3921a25"
server_key_version: "encryptionkey@0"
I20260811 05:13:21.047209 10496 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 05:13:21.049813 10515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:21.051131 10496 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:21.051285 10496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "ade76aa26d02404682b0798c158915aa"
format_stamp: "Formatted at 2026-08-11 05:13:21 on dist-test-slave-04bb"
server_key: "db6317aefa244aee758b02eac598c2d7"
server_key_iv: "7291128d72f0b1e708dcb2bda3921a25"
server_key_version: "encryptionkey@0"
I20260811 05:13:21.051393 10496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:21.069439 10496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:21.072867 10496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:21.073078 10496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:21.074642 10496 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:21.074695 10496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:21.074736 10496 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:21.074792 10496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 05:13:21 dist-test-slave-04bb krb5kdc[10103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425201, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:21.086112 10496 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:39631
I20260811 05:13:21.086175 10703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:39631 every 8 connection(s)
I20260811 05:13:21.087000 10496 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:21.089707 10240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.075582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:48849 (local address 127.7.67.190:35425)
0811 05:13:21.075717 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:21.075720 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:21.076839 (+ 1119us) server_negotiation.cc:407] Connection header received
0811 05:13:21.078019 (+ 1180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:21.078023 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:21.078097 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:21.078201 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:21.079859 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.080829 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.082166 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.082576 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.082725 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.085617 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:21.085622 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:21.085626 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:21.085657 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:21.087385 (+ 1728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:21.088064 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:21.088067 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:21.088068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:21.088152 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:21.088510 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:21.088514 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:21.088516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:21.088734 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:21.088853 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:21.089405 (+ 552us) server_negotiation.cc:299] Negotiation successful
0811 05:13:21.089536 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 05:13:21.090420 10532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.076146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:35425 (local address 127.7.67.130:48849)
0811 05:13:21.076684 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:21.076726 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:21.077748 (+ 1022us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:21.078521 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:21.078530 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:21.078986 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:21.079677 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:21.079698 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.080974 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.080979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:21.082025 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:21.082036 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.082757 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.082764 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:21.082945 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.083542 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:21.083558 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:21.085442 (+ 1884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:21.087553 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:21.087559 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:21.087574 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:21.087955 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:21.088243 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:21.088245 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:21.088247 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:21.088376 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:21.088851 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:21.088858 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:21.089088 (+ 230us) client_negotiation.cc:769] Sending connection context
0811 05:13:21.089345 (+ 257us) client_negotiation.cc:240] Negotiation successful
0811 05:13:21.089647 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":141,"threads_started":1}
I20260811 05:13:21.091816 10705 heartbeater.cc:347] Connected to a master server at 127.7.67.190:35425
I20260811 05:13:21.091998 10705 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:21.092619 10705 heartbeater.cc:511] Master 127.7.67.190:35425 requested a full tablet report, sending...
I20260811 05:13:21.093734 10167 ts_manager.cc:194] Registered new tserver with Master: ade76aa26d02404682b0798c158915aa (127.7.67.130:39631)
I20260811 05:13:21.094493 10167 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.130@KRBTEST.COM'} at 127.7.67.130:48849
I20260811 05:13:21.095748 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 10496
I20260811 05:13:21.095866 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:13:21.096176 7438 external_mini_cluster.cc:1428] Setting key f1493d84d00e60c45fa128c0efb2e8fd
I20260811 05:13:21.100940 10240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.093938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:33129 (local address 127.7.67.190:35425)
0811 05:13:21.094039 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:21.094043 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:21.094263 (+ 220us) server_negotiation.cc:407] Connection header received
0811 05:13:21.094341 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:21.094344 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:21.094386 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:21.094458 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:21.094969 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.095598 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.096578 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.096818 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.096935 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.097962 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:21.097966 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:21.097968 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:21.097998 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:21.099243 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:21.099691 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:21.099702 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:21.099703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:21.099783 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:21.100070 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:21.100074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:21.100076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:21.100283 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:21.100397 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:21.100680 (+ 283us) server_negotiation.cc:299] Negotiation successful
0811 05:13:21.100806 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 05:13:21.100947 10713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.093887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:35425 (local address 127.7.67.130:33129)
0811 05:13:21.094154 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:21.094168 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:21.094260 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:21.094490 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:21.094491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:21.094691 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:21.094847 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:21.094859 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.095726 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.095730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:21.096437 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:21.096448 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.096933 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.096936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:21.097006 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.097494 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:21.097507 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:21.097824 (+ 317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:21.099355 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:21.099359 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:21.099361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:21.099585 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:21.099883 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:21.099886 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:21.099888 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:21.099975 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:21.100388 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:21.100391 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:21.100486 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 05:13:21.100623 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 05:13:21.100767 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":113,"threads_started":1}
WARNING: no policy specified for kudu/127.7.67.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
I20260811 05:13:21.125403 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:35425
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:35049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:21.241657 10719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:21.241919 10719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:21.241966 10719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:21.245998 10719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:21.246054 10719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:21.246093 10719 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 05:13:21.246107 10719 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:13:21.246140 10719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:13:21.251523 10719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:35049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.67.190:35425
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:21.252759 10719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:21.253763 10719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:21.263839 10727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:21.263953 10719 server_base.cc:1034] running on GCE node
W20260811 05:13:21.263827 10730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:21.263827 10726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:21.264604 10719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:21.265300 10719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:21.266510 10719 hybrid_clock.cc:648] HybridClock initialized: now 1786425201266477 us; error 43 us; skew 500 ppm
Aug 11 05:13:21 dist-test-slave-04bb krb5kdc[10103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425201, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:21.269690 10719 init.cc:377] Logged in from keytab as kudu/127.7.67.131@KRBTEST.COM (short username kudu)
I20260811 05:13:21.270598 10719 webserver.cc:460] Webserver started at http://127.7.67.131:43291/ using document root <none> and password file <none>
I20260811 05:13:21.271229 10719 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:21.271327 10719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:21.271569 10719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:21.273705 10719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "1c87338c31514f789c4a801ba48c0595"
format_stamp: "Formatted at 2026-08-11 05:13:21 on dist-test-slave-04bb"
server_key: "3ad0fbcd2bba5854707ca093431d3ab4"
server_key_iv: "653d35c377213ccb4967f27d8ba3a42a"
server_key_version: "encryptionkey@0"
I20260811 05:13:21.274289 10719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "1c87338c31514f789c4a801ba48c0595"
format_stamp: "Formatted at 2026-08-11 05:13:21 on dist-test-slave-04bb"
server_key: "3ad0fbcd2bba5854707ca093431d3ab4"
server_key_iv: "653d35c377213ccb4967f27d8ba3a42a"
server_key_version: "encryptionkey@0"
I20260811 05:13:21.278002 10719 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:13:21.280362 10739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:21.281701 10719 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:13:21.281844 10719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "1c87338c31514f789c4a801ba48c0595"
format_stamp: "Formatted at 2026-08-11 05:13:21 on dist-test-slave-04bb"
server_key: "3ad0fbcd2bba5854707ca093431d3ab4"
server_key_iv: "653d35c377213ccb4967f27d8ba3a42a"
server_key_version: "encryptionkey@0"
I20260811 05:13:21.281953 10719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:21.302690 10719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:21.306327 10719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:21.306586 10719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:21.308158 10719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:21.308236 10719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:21.308336 10719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:21.308387 10719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 05:13:21 dist-test-slave-04bb krb5kdc[10103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425201, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:21.319193 10719 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:38291
I20260811 05:13:21.319252 10925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:38291 every 8 connection(s)
I20260811 05:13:21.320241 10719 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:13:21.322386 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 10719
I20260811 05:13:21.322535 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:13:21.322922 7438 external_mini_cluster.cc:1428] Setting key 10fad1e70190727e5a568ab96937109e
I20260811 05:13:21.324478 10240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.309186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:55677 (local address 127.7.67.190:35425)
0811 05:13:21.309335 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:21.309339 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:21.310429 (+ 1090us) server_negotiation.cc:407] Connection header received
0811 05:13:21.311721 (+ 1292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:21.311727 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:21.311812 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:21.311918 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:21.313748 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.314657 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.316457 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.316748 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.316868 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.319944 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:21.319947 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:21.319949 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:21.320003 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:21.321882 (+ 1879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:21.322560 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:21.322564 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:21.322566 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:21.322659 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:21.323089 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:21.323092 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:21.323093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:21.323345 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:21.323453 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:21.324186 (+ 733us) server_negotiation.cc:299] Negotiation successful
0811 05:13:21.324312 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 05:13:21.325290 10762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.309744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:35425 (local address 127.7.67.131:55677)
0811 05:13:21.310269 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:21.310313 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:21.311364 (+ 1051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:21.312274 (+ 910us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:21.312287 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:21.312835 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:21.313583 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:21.313609 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.314827 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.314830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:21.316316 (+ 1486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:21.316326 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.316910 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.316914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:21.317109 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.317824 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:21.317845 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:21.319742 (+ 1897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:21.322012 (+ 2270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:21.322018 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:21.322032 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:21.322399 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:21.322792 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:21.322796 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:21.322798 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:21.322990 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:21.323629 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:21.323634 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:21.323943 (+ 309us) client_negotiation.cc:769] Sending connection context
0811 05:13:21.324141 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 05:13:21.324457 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":140,"threads_started":1}
I20260811 05:13:21.326328 10926 heartbeater.cc:347] Connected to a master server at 127.7.67.190:35425
I20260811 05:13:21.326465 10926 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:21.327014 10926 heartbeater.cc:511] Master 127.7.67.190:35425 requested a full tablet report, sending...
I20260811 05:13:21.328171 10171 ts_manager.cc:194] Registered new tserver with Master: 1c87338c31514f789c4a801ba48c0595 (127.7.67.131:38291)
I20260811 05:13:21.329058 10171 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.131@KRBTEST.COM'} at 127.7.67.131:55677
I20260811 05:13:21.336071 10240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.328827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:33011 (local address 127.7.67.190:35425)
0811 05:13:21.328950 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:21.328955 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:21.329174 (+ 219us) server_negotiation.cc:407] Connection header received
0811 05:13:21.329294 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:21.329297 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:21.329338 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:21.329432 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:21.330014 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.330741 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.331579 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.331795 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.331874 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.332923 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:21.332927 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:21.332930 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:21.332958 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:21.334436 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:21.334853 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:21.334856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:21.334858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:21.334943 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:21.335243 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:21.335246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:21.335247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:21.335429 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:21.335575 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:21.335830 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 05:13:21.335937 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 05:13:21.336086 10933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.328760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:35425 (local address 127.7.67.131:33011)
0811 05:13:21.329088 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:21.329100 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:21.329212 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:21.329537 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:21.329541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:21.329750 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:21.329891 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:21.329897 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.330882 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.330885 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:21.331465 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:21.331474 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.331883 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.331886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:21.331953 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.332405 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:21.332416 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:21.332789 (+ 373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:21.334536 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:21.334538 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:21.334540 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:21.334740 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:21.335040 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:21.335042 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:21.335044 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:21.335122 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:21.335570 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:21.335572 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:21.335649 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 05:13:21.335767 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 05:13:21.335904 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":5,"thread_start_us":101,"threads_started":1}
I20260811 05:13:21.337044 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 05:13:21 dist-test-slave-04bb krb5kdc[10103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 05:13:21 dist-test-slave-04bb krb5kdc[10103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425201, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:21.364250 10240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.355870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50328 (local address 127.7.67.190:35425)
0811 05:13:21.355980 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:21.355983 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:21.355996 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:13:21.356036 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:21.356038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:21.356077 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:21.356151 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:21.356958 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.357511 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.358364 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.358580 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.358694 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.361162 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:21.361165 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:21.361167 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:21.361189 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:21.362538 (+ 1349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:21.362984 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:21.362989 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:21.362991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:21.363073 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:21.363370 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:21.363374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:21.363376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:21.363555 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:21.363702 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:21.363951 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 05:13:21.364066 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 05:13:21.367244 10167 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50328:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:13:21.370695 10167 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:21.385479 10957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.379255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43134 (local address 127.7.67.131:38291)
0811 05:13:21.380826 (+ 1571us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:21.380839 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:21.380862 (+ 23us) server_negotiation.cc:407] Connection header received
0811 05:13:21.380941 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:21.380947 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:21.381117 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:21.381291 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:21.381744 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.382615 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.383983 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.384788 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.384913 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.385086 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:21.385244 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 05:13:21.385337 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1467,"thread_start_us":109,"threads_started":1}
I20260811 05:13:21.385471 10961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.379097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:38291 (local address 127.0.0.1:43134)
0811 05:13:21.380317 (+ 1220us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:21.380372 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:21.380508 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:21.381428 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:21.381431 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:21.381464 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:21.381620 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:21.381625 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.382769 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.382773 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:21.383854 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:21.383864 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.384912 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.384916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:21.385021 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.385058 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 05:13:21.385194 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 05:13:21.385321 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1041,"mutex_wait_us":37,"spinlock_wait_cycles":23040,"thread_start_us":105,"threads_started":1}
I20260811 05:13:21.385972 10960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.379315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:43129 (local address 127.0.0.1:60076)
0811 05:13:21.380316 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:21.380372 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:21.380508 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:21.381264 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:21.381271 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:21.381294 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:21.381440 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:21.381446 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.382881 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.382884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:21.384342 (+ 1458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:21.384352 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.385367 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.385371 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:21.385466 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.385498 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 05:13:21.385551 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 05:13:21.385736 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":941,"mutex_wait_us":45,"spinlock_wait_cycles":7168,"thread_start_us":97,"threads_started":1}
I20260811 05:13:21.386360 10956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.379380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:39631 (local address 127.0.0.1:49018)
0811 05:13:21.380770 (+ 1390us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:21.380783 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:21.380880 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:21.381422 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:21.381425 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:21.381461 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:21.381633 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:21.381640 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.382982 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.382985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:21.384249 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:21.384259 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.385931 (+ 1672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.385935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:21.386035 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.386067 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 05:13:21.386145 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 05:13:21.386237 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1327,"mutex_wait_us":100,"spinlock_wait_cycles":2176,"thread_start_us":86,"threads_started":1}
I20260811 05:13:21.386775 10963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.379934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60076 (local address 127.7.67.129:43129)
0811 05:13:21.380750 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:21.380755 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:21.380776 (+ 21us) server_negotiation.cc:407] Connection header received
0811 05:13:21.380873 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:21.380878 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:21.381088 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:21.381272 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:21.381574 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.382744 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.384445 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.385223 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.386241 (+ 1018us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.386382 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:21.386525 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 05:13:21.386636 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":729,"thread_start_us":77,"threads_started":1}
I20260811 05:13:21.387176 10964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.379909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49018 (local address 127.7.67.130:39631)
0811 05:13:21.380536 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:21.380584 (+ 48us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:21.380884 (+ 300us) server_negotiation.cc:407] Connection header received
0811 05:13:21.380980 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:21.380984 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:21.381126 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:21.381271 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:21.381779 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.382869 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.384375 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.385214 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.385343 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.385508 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:21.386613 (+ 1105us) server_negotiation.cc:299] Negotiation successful
0811 05:13:21.386699 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":112,"threads_started":1}
I20260811 05:13:21.388803 10818 tablet_service.cc:1467] Processing CreateTablet for tablet 4c197b05bf284d4c892e3c476cc4cb07 (DEFAULT_TABLE table=test-table [id=4abfea41b5104ab4ae533abdfe8f202e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:21.389460 10379 tablet_service.cc:1467] Processing CreateTablet for tablet 4c197b05bf284d4c892e3c476cc4cb07 (DEFAULT_TABLE table=test-table [id=4abfea41b5104ab4ae533abdfe8f202e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:21.389973 10818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c197b05bf284d4c892e3c476cc4cb07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:21.389860 10595 tablet_service.cc:1467] Processing CreateTablet for tablet 4c197b05bf284d4c892e3c476cc4cb07 (DEFAULT_TABLE table=test-table [id=4abfea41b5104ab4ae533abdfe8f202e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:21.390440 10379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c197b05bf284d4c892e3c476cc4cb07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:21.390832 10595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c197b05bf284d4c892e3c476cc4cb07. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:21.396575 10965 tablet_bootstrap.cc:492] T 4c197b05bf284d4c892e3c476cc4cb07 P 220563d55ccc4fac99d9930a81a07316: Bootstrap starting.
I20260811 05:13:21.398027 10967 tablet_bootstrap.cc:492] T 4c197b05bf284d4c892e3c476cc4cb07 P ade76aa26d02404682b0798c158915aa: Bootstrap starting.
I20260811 05:13:21.398780 10966 tablet_bootstrap.cc:492] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595: Bootstrap starting.
I20260811 05:13:21.399281 10965 tablet_bootstrap.cc:654] T 4c197b05bf284d4c892e3c476cc4cb07 P 220563d55ccc4fac99d9930a81a07316: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:21.400195 10965 log.cc:824] T 4c197b05bf284d4c892e3c476cc4cb07 P 220563d55ccc4fac99d9930a81a07316: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:21.400993 10966 tablet_bootstrap.cc:654] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:21.401813 10966 log.cc:824] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:21.402349 10965 tablet_bootstrap.cc:492] T 4c197b05bf284d4c892e3c476cc4cb07 P 220563d55ccc4fac99d9930a81a07316: No bootstrap required, opened a new log
I20260811 05:13:21.402508 10967 tablet_bootstrap.cc:654] T 4c197b05bf284d4c892e3c476cc4cb07 P ade76aa26d02404682b0798c158915aa: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:21.402560 10965 ts_tablet_manager.cc:1397] T 4c197b05bf284d4c892e3c476cc4cb07 P 220563d55ccc4fac99d9930a81a07316: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260811 05:13:21.403290 10967 log.cc:824] T 4c197b05bf284d4c892e3c476cc4cb07 P ade76aa26d02404682b0798c158915aa: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:21.403548 10966 tablet_bootstrap.cc:492] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595: No bootstrap required, opened a new log
I20260811 05:13:21.403746 10966 ts_tablet_manager.cc:1397] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 05:13:21.406215 10967 tablet_bootstrap.cc:492] T 4c197b05bf284d4c892e3c476cc4cb07 P ade76aa26d02404682b0798c158915aa: No bootstrap required, opened a new log
I20260811 05:13:21.406399 10967 ts_tablet_manager.cc:1397] T 4c197b05bf284d4c892e3c476cc4cb07 P ade76aa26d02404682b0798c158915aa: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260811 05:13:21.406839 10965 raft_consensus.cc:348] T 4c197b05bf284d4c892e3c476cc4cb07 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } } peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } }
I20260811 05:13:21.407105 10965 raft_consensus.cc:374] T 4c197b05bf284d4c892e3c476cc4cb07 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:21.407161 10965 raft_consensus.cc:729] T 4c197b05bf284d4c892e3c476cc4cb07 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 220563d55ccc4fac99d9930a81a07316, State: Initialized, Role: FOLLOWER
I20260811 05:13:21.407011 10966 raft_consensus.cc:348] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } } peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } }
I20260811 05:13:21.407315 10966 raft_consensus.cc:374] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:21.407369 10966 raft_consensus.cc:729] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c87338c31514f789c4a801ba48c0595, State: Initialized, Role: FOLLOWER
I20260811 05:13:21.407642 10965 consensus_queue.cc:260] T 4c197b05bf284d4c892e3c476cc4cb07 P 220563d55ccc4fac99d9930a81a07316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } } peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } }
I20260811 05:13:21.407791 10966 consensus_queue.cc:260] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } } peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } }
I20260811 05:13:21.408703 10965 ts_tablet_manager.cc:1428] T 4c197b05bf284d4c892e3c476cc4cb07 P 220563d55ccc4fac99d9930a81a07316: Time spent starting tablet: real 0.006s user 0.006s sys 0.002s
I20260811 05:13:21.408856 10966 ts_tablet_manager.cc:1428] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 05:13:21.409056 10483 heartbeater.cc:503] Master 127.7.67.190:35425 was elected leader, sending a full tablet report...
I20260811 05:13:21.409080 10926 heartbeater.cc:503] Master 127.7.67.190:35425 was elected leader, sending a full tablet report...
I20260811 05:13:21.410341 10967 raft_consensus.cc:348] T 4c197b05bf284d4c892e3c476cc4cb07 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } } peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } }
I20260811 05:13:21.410595 10967 raft_consensus.cc:374] T 4c197b05bf284d4c892e3c476cc4cb07 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:21.410665 10967 raft_consensus.cc:729] T 4c197b05bf284d4c892e3c476cc4cb07 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ade76aa26d02404682b0798c158915aa, State: Initialized, Role: FOLLOWER
I20260811 05:13:21.411087 10967 consensus_queue.cc:260] T 4c197b05bf284d4c892e3c476cc4cb07 P ade76aa26d02404682b0798c158915aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } } peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } }
I20260811 05:13:21.411773 10705 heartbeater.cc:503] Master 127.7.67.190:35425 was elected leader, sending a full tablet report...
I20260811 05:13:21.412062 10967 ts_tablet_manager.cc:1428] T 4c197b05bf284d4c892e3c476cc4cb07 P ade76aa26d02404682b0798c158915aa: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 05:13:21.563634 10974 raft_consensus.cc:482] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:21.563915 10974 raft_consensus.cc:504] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } } peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } }
I20260811 05:13:21.565188 10974 leader_election.cc:288] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 220563d55ccc4fac99d9930a81a07316 (127.7.67.129:43129), ade76aa26d02404682b0798c158915aa (127.7.67.130:39631)
I20260811 05:13:21.569448 10762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.565508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:39631 (local address 127.7.67.131:45973)
0811 05:13:21.565677 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:21.565695 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:21.565805 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:21.566177 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:21.566180 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:21.566202 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:21.566346 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:21.566356 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.567206 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.567212 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:21.568224 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:21.568235 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.568978 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.568981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:21.569097 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.569125 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 05:13:21.569212 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 05:13:21.569310 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 05:13:21.569494 10964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.565579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:45973 (local address 127.7.67.130:39631)
0811 05:13:21.565716 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:21.565722 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:21.565748 (+ 26us) server_negotiation.cc:407] Connection header received
0811 05:13:21.565944 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:21.565947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:21.566006 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:21.566086 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:21.566491 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.567067 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.568352 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.568857 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.568960 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.569003 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:21.569285 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 05:13:21.569365 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 05:13:21.570142 10627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c197b05bf284d4c892e3c476cc4cb07" candidate_uuid: "1c87338c31514f789c4a801ba48c0595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ade76aa26d02404682b0798c158915aa" is_pre_election: true
I20260811 05:13:21.570317 10976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.565510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:43129 (local address 127.7.67.131:38857)
0811 05:13:21.566201 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:21.566215 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:21.566313 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:21.566590 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:21.566593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:21.566612 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:21.567275 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:21.567284 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.568005 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.568008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:21.568962 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:21.568971 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.569914 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.569918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:21.570006 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.570027 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 05:13:21.570109 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 05:13:21.570177 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":616,"mutex_wait_us":279,"thread_start_us":183,"threads_started":1}
I20260811 05:13:21.570353 10963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.565636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:38857 (local address 127.7.67.129:43129)
0811 05:13:21.565775 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:21.565780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:21.566313 (+ 533us) server_negotiation.cc:407] Connection header received
0811 05:13:21.566408 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:21.566413 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:21.566468 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:21.566574 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:21.567408 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.567889 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.569106 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.569786 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.569906 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.569943 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:21.570133 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 05:13:21.570223 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 05:13:21.570497 10627 raft_consensus.cc:2454] T 4c197b05bf284d4c892e3c476cc4cb07 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c87338c31514f789c4a801ba48c0595 in term 0.
I20260811 05:13:21.571029 10745 leader_election.cc:302] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c87338c31514f789c4a801ba48c0595, ade76aa26d02404682b0798c158915aa; no voters:
I20260811 05:13:21.571076 10406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c197b05bf284d4c892e3c476cc4cb07" candidate_uuid: "1c87338c31514f789c4a801ba48c0595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "220563d55ccc4fac99d9930a81a07316" is_pre_election: true
W20260811 05:13:21.571326 10927 tablet.cc:2369] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:13:21.571401 10406 raft_consensus.cc:2454] T 4c197b05bf284d4c892e3c476cc4cb07 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c87338c31514f789c4a801ba48c0595 in term 0.
I20260811 05:13:21.571338 10974 raft_consensus.cc:2781] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:21.571491 10974 raft_consensus.cc:482] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:21.571548 10974 raft_consensus.cc:3037] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:21.572742 10974 raft_consensus.cc:504] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } } peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } }
I20260811 05:13:21.573105 10974 leader_election.cc:288] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595 [CANDIDATE]: Term 1 election: Requested vote from peers 220563d55ccc4fac99d9930a81a07316 (127.7.67.129:43129), ade76aa26d02404682b0798c158915aa (127.7.67.130:39631)
I20260811 05:13:21.573477 10406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c197b05bf284d4c892e3c476cc4cb07" candidate_uuid: "1c87338c31514f789c4a801ba48c0595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "220563d55ccc4fac99d9930a81a07316"
I20260811 05:13:21.573487 10627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c197b05bf284d4c892e3c476cc4cb07" candidate_uuid: "1c87338c31514f789c4a801ba48c0595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ade76aa26d02404682b0798c158915aa"
I20260811 05:13:21.573577 10406 raft_consensus.cc:3037] T 4c197b05bf284d4c892e3c476cc4cb07 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:21.573618 10627 raft_consensus.cc:3037] T 4c197b05bf284d4c892e3c476cc4cb07 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:21.574676 10406 raft_consensus.cc:2454] T 4c197b05bf284d4c892e3c476cc4cb07 P 220563d55ccc4fac99d9930a81a07316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c87338c31514f789c4a801ba48c0595 in term 1.
I20260811 05:13:21.574702 10627 raft_consensus.cc:2454] T 4c197b05bf284d4c892e3c476cc4cb07 P ade76aa26d02404682b0798c158915aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c87338c31514f789c4a801ba48c0595 in term 1.
I20260811 05:13:21.575011 10744 leader_election.cc:302] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c87338c31514f789c4a801ba48c0595, 220563d55ccc4fac99d9930a81a07316; no voters:
I20260811 05:13:21.575255 10974 raft_consensus.cc:2781] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:21.575486 10974 raft_consensus.cc:686] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595 [term 1 LEADER]: Becoming Leader. State: Replica: 1c87338c31514f789c4a801ba48c0595, State: Running, Role: LEADER
I20260811 05:13:21.575834 10974 consensus_queue.cc:237] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } } peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } }
I20260811 05:13:21.578969 10170 catalog_manager.cc:5495] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c87338c31514f789c4a801ba48c0595 (127.7.67.131). New cstate: current_term: 1 leader_uuid: "1c87338c31514f789c4a801ba48c0595" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } health_report { overall_health: UNKNOWN } } }
W20260811 05:13:21.588220 10706 tablet.cc:2369] T 4c197b05bf284d4c892e3c476cc4cb07 P ade76aa26d02404682b0798c158915aa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:13:21.615621 10484 tablet.cc:2369] T 4c197b05bf284d4c892e3c476cc4cb07 P 220563d55ccc4fac99d9930a81a07316: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 05:13:21 dist-test-slave-04bb krb5kdc[10103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425200, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:21.651592 10240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.642427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50330 (local address 127.7.67.190:35425)
0811 05:13:21.642583 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:21.642589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:21.642772 (+ 183us) server_negotiation.cc:407] Connection header received
0811 05:13:21.642956 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:21.642965 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:21.643005 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:21.643097 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:21.643798 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.644399 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.645297 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.645526 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.645646 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.648019 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:21.648022 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:21.648024 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:21.648047 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:21.649561 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:21.650089 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:21.650095 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:21.650097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:21.650217 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:21.650525 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:21.650530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:21.650532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:21.650777 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:21.650897 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:21.651233 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 05:13:21.651454 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":95,"server-negotiator.queue_time_us":52}
I20260811 05:13:21.651593 10993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.642335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:35425 (local address 127.0.0.1:50330)
0811 05:13:21.642666 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:21.642679 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:21.642830 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:21.643121 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:21.643124 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:21.643563 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:21.643660 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:21.643666 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.644546 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.644551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:21.645166 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:21.645175 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.645647 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.645651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:21.645747 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.646337 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:21.646350 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:21.647847 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:21.649707 (+ 1860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:21.649718 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:21.649725 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:21.649966 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:21.650325 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:21.650331 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:21.650333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:21.650423 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:21.650909 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:21.650918 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:21.651040 (+ 122us) client_negotiation.cc:769] Sending connection context
0811 05:13:21.651172 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 05:13:21.651372 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":115,"threads_started":1}
I20260811 05:13:21.656212 10170 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:50330:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 05:13:21.656965 10170 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:21.662338 10379 tablet_service.cc:1467] Processing CreateTablet for tablet d641cce1ed1740e591b60d1d3e59e9c5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=212c8f80179d465eb6995b34bc3840dd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:13:21.662528 10818 tablet_service.cc:1467] Processing CreateTablet for tablet d641cce1ed1740e591b60d1d3e59e9c5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=212c8f80179d465eb6995b34bc3840dd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:13:21.662462 10595 tablet_service.cc:1467] Processing CreateTablet for tablet d641cce1ed1740e591b60d1d3e59e9c5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=212c8f80179d465eb6995b34bc3840dd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:13:21.663074 10379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d641cce1ed1740e591b60d1d3e59e9c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:21.663201 10818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d641cce1ed1740e591b60d1d3e59e9c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:21.663213 10595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d641cce1ed1740e591b60d1d3e59e9c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:21.665863 10965 tablet_bootstrap.cc:492] T d641cce1ed1740e591b60d1d3e59e9c5 P 220563d55ccc4fac99d9930a81a07316: Bootstrap starting.
I20260811 05:13:21.666385 10966 tablet_bootstrap.cc:492] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595: Bootstrap starting.
I20260811 05:13:21.666648 10967 tablet_bootstrap.cc:492] T d641cce1ed1740e591b60d1d3e59e9c5 P ade76aa26d02404682b0798c158915aa: Bootstrap starting.
I20260811 05:13:21.666921 10965 tablet_bootstrap.cc:654] T d641cce1ed1740e591b60d1d3e59e9c5 P 220563d55ccc4fac99d9930a81a07316: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:21.667373 10966 tablet_bootstrap.cc:654] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:21.667706 10967 tablet_bootstrap.cc:654] T d641cce1ed1740e591b60d1d3e59e9c5 P ade76aa26d02404682b0798c158915aa: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:21.668516 10965 tablet_bootstrap.cc:492] T d641cce1ed1740e591b60d1d3e59e9c5 P 220563d55ccc4fac99d9930a81a07316: No bootstrap required, opened a new log
I20260811 05:13:21.668635 10965 ts_tablet_manager.cc:1397] T d641cce1ed1740e591b60d1d3e59e9c5 P 220563d55ccc4fac99d9930a81a07316: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260811 05:13:21.669098 10967 tablet_bootstrap.cc:492] T d641cce1ed1740e591b60d1d3e59e9c5 P ade76aa26d02404682b0798c158915aa: No bootstrap required, opened a new log
I20260811 05:13:21.669098 10966 tablet_bootstrap.cc:492] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595: No bootstrap required, opened a new log
I20260811 05:13:21.669193 10966 ts_tablet_manager.cc:1397] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:13:21.669202 10967 ts_tablet_manager.cc:1397] T d641cce1ed1740e591b60d1d3e59e9c5 P ade76aa26d02404682b0798c158915aa: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:13:21.669121 10965 raft_consensus.cc:348] T d641cce1ed1740e591b60d1d3e59e9c5 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:21.669282 10965 raft_consensus.cc:374] T d641cce1ed1740e591b60d1d3e59e9c5 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:21.669332 10965 raft_consensus.cc:729] T d641cce1ed1740e591b60d1d3e59e9c5 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 220563d55ccc4fac99d9930a81a07316, State: Initialized, Role: FOLLOWER
I20260811 05:13:21.669512 10965 consensus_queue.cc:260] T d641cce1ed1740e591b60d1d3e59e9c5 P 220563d55ccc4fac99d9930a81a07316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:21.669812 10965 ts_tablet_manager.cc:1428] T d641cce1ed1740e591b60d1d3e59e9c5 P 220563d55ccc4fac99d9930a81a07316: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 05:13:21.669762 10966 raft_consensus.cc:348] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:21.669878 10966 raft_consensus.cc:374] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:21.669925 10966 raft_consensus.cc:729] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c87338c31514f789c4a801ba48c0595, State: Initialized, Role: FOLLOWER
I20260811 05:13:21.670056 10966 consensus_queue.cc:260] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:21.670343 10966 ts_tablet_manager.cc:1428] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 05:13:21.670467 10971 tablet_replica.cc:439] T d641cce1ed1740e591b60d1d3e59e9c5 P 220563d55ccc4fac99d9930a81a07316: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } } }
I20260811 05:13:21.670567 10974 tablet_replica.cc:439] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } } }
I20260811 05:13:21.670630 10971 tablet_replica.cc:442] T d641cce1ed1740e591b60d1d3e59e9c5 P 220563d55ccc4fac99d9930a81a07316: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:21.670686 10974 tablet_replica.cc:442] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:21.671229 10967 raft_consensus.cc:348] T d641cce1ed1740e591b60d1d3e59e9c5 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:21.671362 10967 raft_consensus.cc:374] T d641cce1ed1740e591b60d1d3e59e9c5 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:21.671412 10967 raft_consensus.cc:729] T d641cce1ed1740e591b60d1d3e59e9c5 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ade76aa26d02404682b0798c158915aa, State: Initialized, Role: FOLLOWER
I20260811 05:13:21.671545 10967 consensus_queue.cc:260] T d641cce1ed1740e591b60d1d3e59e9c5 P ade76aa26d02404682b0798c158915aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:21.671859 10967 ts_tablet_manager.cc:1428] T d641cce1ed1740e591b60d1d3e59e9c5 P ade76aa26d02404682b0798c158915aa: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260811 05:13:21.671814 10975 tablet_replica.cc:439] T d641cce1ed1740e591b60d1d3e59e9c5 P ade76aa26d02404682b0798c158915aa: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } } }
I20260811 05:13:21.671954 10975 tablet_replica.cc:442] T d641cce1ed1740e591b60d1d3e59e9c5 P ade76aa26d02404682b0798c158915aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:21.687028 10974 raft_consensus.cc:482] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:21.687127 10974 raft_consensus.cc:504] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:21.687517 10974 leader_election.cc:288] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 220563d55ccc4fac99d9930a81a07316 (127.7.67.129:43129), ade76aa26d02404682b0798c158915aa (127.7.67.130:39631)
I20260811 05:13:21.687897 10406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d641cce1ed1740e591b60d1d3e59e9c5" candidate_uuid: "1c87338c31514f789c4a801ba48c0595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "220563d55ccc4fac99d9930a81a07316" is_pre_election: true
I20260811 05:13:21.687928 10627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d641cce1ed1740e591b60d1d3e59e9c5" candidate_uuid: "1c87338c31514f789c4a801ba48c0595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ade76aa26d02404682b0798c158915aa" is_pre_election: true
I20260811 05:13:21.688050 10406 raft_consensus.cc:2454] T d641cce1ed1740e591b60d1d3e59e9c5 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c87338c31514f789c4a801ba48c0595 in term 0.
I20260811 05:13:21.688087 10627 raft_consensus.cc:2454] T d641cce1ed1740e591b60d1d3e59e9c5 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c87338c31514f789c4a801ba48c0595 in term 0.
I20260811 05:13:21.688390 10744 leader_election.cc:302] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1c87338c31514f789c4a801ba48c0595, 220563d55ccc4fac99d9930a81a07316, ade76aa26d02404682b0798c158915aa; no voters:
I20260811 05:13:21.688581 10974 raft_consensus.cc:2781] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:21.688643 10974 raft_consensus.cc:482] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:21.688689 10974 raft_consensus.cc:3037] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:21.689533 10974 raft_consensus.cc:504] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:21.689843 10974 leader_election.cc:288] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595 [CANDIDATE]: Term 1 election: Requested vote from peers 220563d55ccc4fac99d9930a81a07316 (127.7.67.129:43129), ade76aa26d02404682b0798c158915aa (127.7.67.130:39631)
I20260811 05:13:21.690137 10406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d641cce1ed1740e591b60d1d3e59e9c5" candidate_uuid: "1c87338c31514f789c4a801ba48c0595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "220563d55ccc4fac99d9930a81a07316"
I20260811 05:13:21.690194 10627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d641cce1ed1740e591b60d1d3e59e9c5" candidate_uuid: "1c87338c31514f789c4a801ba48c0595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ade76aa26d02404682b0798c158915aa"
I20260811 05:13:21.690303 10406 raft_consensus.cc:3037] T d641cce1ed1740e591b60d1d3e59e9c5 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:21.690312 10627 raft_consensus.cc:3037] T d641cce1ed1740e591b60d1d3e59e9c5 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:21.691160 10627 raft_consensus.cc:2454] T d641cce1ed1740e591b60d1d3e59e9c5 P ade76aa26d02404682b0798c158915aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c87338c31514f789c4a801ba48c0595 in term 1.
I20260811 05:13:21.691169 10406 raft_consensus.cc:2454] T d641cce1ed1740e591b60d1d3e59e9c5 P 220563d55ccc4fac99d9930a81a07316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c87338c31514f789c4a801ba48c0595 in term 1.
I20260811 05:13:21.691435 10745 leader_election.cc:302] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c87338c31514f789c4a801ba48c0595, ade76aa26d02404682b0798c158915aa; no voters:
I20260811 05:13:21.691619 10974 raft_consensus.cc:2781] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:21.691741 10974 raft_consensus.cc:686] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595 [term 1 LEADER]: Becoming Leader. State: Replica: 1c87338c31514f789c4a801ba48c0595, State: Running, Role: LEADER
I20260811 05:13:21.691861 10974 consensus_queue.cc:237] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:21.692639 10977 tablet_replica.cc:439] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595: TxnStatusTablet state changed. Reason: New leader 1c87338c31514f789c4a801ba48c0595. Latest consensus state: current_term: 1 leader_uuid: "1c87338c31514f789c4a801ba48c0595" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } } }
I20260811 05:13:21.692756 10977 tablet_replica.cc:442] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:13:21.693073 11000 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 05:13:21.693305 11000 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 05:13:21.693643 10171 catalog_manager.cc:5495] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c87338c31514f789c4a801ba48c0595 (127.7.67.131). New cstate: current_term: 1 leader_uuid: "1c87338c31514f789c4a801ba48c0595" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } health_report { overall_health: HEALTHY } } }
I20260811 05:13:21.713251 10993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.709479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:38291 (local address 127.0.0.1:43146)
0811 05:13:21.709616 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:21.709634 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:21.709759 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:21.710103 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:21.710107 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:21.710117 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:13:21.710235 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:21.710241 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.710968 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.710972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:21.711592 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:21.711601 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.712068 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.712072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:21.712187 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.712233 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:13:21.712954 (+ 721us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:13:21.712972 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 05:13:21.713044 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 05:13:21.713133 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 05:13:21.713251 10957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.709583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43146 (local address 127.7.67.131:38291)
0811 05:13:21.709719 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:21.709724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:21.709739 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:13:21.709917 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:21.709921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:21.709982 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:21.710087 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:21.710347 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.710850 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.711726 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.711942 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.712065 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.712387 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:21.712854 (+ 467us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:21.712980 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:21.713053 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 05:13:21.713106 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 05:13:21.719374 10627 raft_consensus.cc:1261] T d641cce1ed1740e591b60d1d3e59e9c5 P ade76aa26d02404682b0798c158915aa [term 1 FOLLOWER]: Refusing update from remote peer 1c87338c31514f789c4a801ba48c0595: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:21.719448 10406 raft_consensus.cc:1261] T d641cce1ed1740e591b60d1d3e59e9c5 P 220563d55ccc4fac99d9930a81a07316 [term 1 FOLLOWER]: Refusing update from remote peer 1c87338c31514f789c4a801ba48c0595: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:21.720101 10977 consensus_queue.cc:1035] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:21.720296 10974 consensus_queue.cc:1035] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595 [LEADER]: Connected to new peer: Peer: permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:21.724426 10971 tablet_replica.cc:439] T d641cce1ed1740e591b60d1d3e59e9c5 P 220563d55ccc4fac99d9930a81a07316: TxnStatusTablet state changed. Reason: New leader 1c87338c31514f789c4a801ba48c0595. Latest consensus state: current_term: 1 leader_uuid: "1c87338c31514f789c4a801ba48c0595" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } } }
I20260811 05:13:21.724584 10971 tablet_replica.cc:442] T d641cce1ed1740e591b60d1d3e59e9c5 P 220563d55ccc4fac99d9930a81a07316: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:21.726468 10974 tablet_replica.cc:439] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1c87338c31514f789c4a801ba48c0595" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } } }
I20260811 05:13:21.726575 10974 tablet_replica.cc:442] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:13:21.726645 10975 tablet_replica.cc:439] T d641cce1ed1740e591b60d1d3e59e9c5 P ade76aa26d02404682b0798c158915aa: TxnStatusTablet state changed. Reason: New leader 1c87338c31514f789c4a801ba48c0595. Latest consensus state: current_term: 1 leader_uuid: "1c87338c31514f789c4a801ba48c0595" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } } }
I20260811 05:13:21.726789 10975 tablet_replica.cc:442] T d641cce1ed1740e591b60d1d3e59e9c5 P ade76aa26d02404682b0798c158915aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:21.726902 10975 tablet_replica.cc:439] T d641cce1ed1740e591b60d1d3e59e9c5 P ade76aa26d02404682b0798c158915aa: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1c87338c31514f789c4a801ba48c0595" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } } }
I20260811 05:13:21.726979 10975 tablet_replica.cc:442] T d641cce1ed1740e591b60d1d3e59e9c5 P ade76aa26d02404682b0798c158915aa: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:21.728739 11005 mvcc.cc:205] Tried to move back new op lower bound from 7317197626235617280 to 7317197626131492864. Current Snapshot: MvccSnapshot[applied={T|T < 7317197626235617280}]
I20260811 05:13:21.732204 10977 tablet_replica.cc:439] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1c87338c31514f789c4a801ba48c0595" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } } }
I20260811 05:13:21.732304 10977 tablet_replica.cc:442] T d641cce1ed1740e591b60d1d3e59e9c5 P 1c87338c31514f789c4a801ba48c0595: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:13:21.734642 10971 tablet_replica.cc:439] T d641cce1ed1740e591b60d1d3e59e9c5 P 220563d55ccc4fac99d9930a81a07316: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1c87338c31514f789c4a801ba48c0595" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } } }
I20260811 05:13:21.734791 10971 tablet_replica.cc:442] T d641cce1ed1740e591b60d1d3e59e9c5 P 220563d55ccc4fac99d9930a81a07316: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:21.734664 11007 mvcc.cc:205] Tried to move back new op lower bound from 7317197626235617280 to 7317197626131492864. Current Snapshot: MvccSnapshot[applied={T|T < 7317197626235617280}]
I20260811 05:13:21.738176 10957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.734619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43156 (local address 127.7.67.131:38291)
0811 05:13:21.734728 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:21.734733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:21.734747 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:13:21.734803 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:21.734808 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:21.734862 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:21.734939 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:21.735275 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.735861 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.737101 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.737304 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.737386 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.737698 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:21.737777 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:21.737874 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:21.738009 (+ 135us) server_negotiation.cc:299] Negotiation successful
0811 05:13:21.738054 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 05:13:21.748270 10957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.745233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:53595 (local address 127.7.67.131:38291)
0811 05:13:21.745381 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:21.745387 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:21.745400 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:13:21.745537 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:21.745540 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:21.745583 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:21.745655 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:21.745890 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.746355 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.747194 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.747393 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.747499 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.747782 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:21.747849 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:21.747938 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:21.748051 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 05:13:21.748131 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":85}
I20260811 05:13:21.748270 10933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:21.745165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:38291 (local address 127.7.67.131:53595)
0811 05:13:21.745333 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:21.745349 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:21.745440 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:21.745674 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:21.745676 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:21.745682 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:13:21.745790 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:21.745795 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.746463 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.746468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:21.747067 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:21.747075 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:21.747499 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:21.747503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:21.747592 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:21.747630 (+ 38us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:13:21.747953 (+ 323us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:13:21.747959 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 05:13:21.748041 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 05:13:21.748153 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":27}
I20260811 05:13:21.755074 10627 raft_consensus.cc:1261] T 4c197b05bf284d4c892e3c476cc4cb07 P ade76aa26d02404682b0798c158915aa [term 1 FOLLOWER]: Refusing update from remote peer 1c87338c31514f789c4a801ba48c0595: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:21.755147 10406 raft_consensus.cc:1261] T 4c197b05bf284d4c892e3c476cc4cb07 P 220563d55ccc4fac99d9930a81a07316 [term 1 FOLLOWER]: Refusing update from remote peer 1c87338c31514f789c4a801ba48c0595: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:21.755549 10977 consensus_queue.cc:1035] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:21.755738 10974 consensus_queue.cc:1035] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595 [LEADER]: Connected to new peer: Peer: permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:21.760829 11003 mvcc.cc:205] Tried to move back new op lower bound from 7317197626384908288 to 7317197625657196544. Current Snapshot: MvccSnapshot[applied={T|T < 7317197626384908288}]
I20260811 05:13:21.795197 10170 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:21.795434 10170 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:21.797792 10170 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115: Sending DeleteTablet for 3 replicas of tablet 4c197b05bf284d4c892e3c476cc4cb07
I20260811 05:13:21.798468 10818 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c197b05bf284d4c892e3c476cc4cb07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:21 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:43134
I20260811 05:13:21.798739 10379 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c197b05bf284d4c892e3c476cc4cb07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:21 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:60076
I20260811 05:13:21.798732 10595 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c197b05bf284d4c892e3c476cc4cb07 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:21 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:49018
I20260811 05:13:21.798905 11034 tablet_replica.cc:330] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595: stopping tablet replica
I20260811 05:13:21.799202 11034 raft_consensus.cc:2229] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:13:21.799317 11037 tablet_replica.cc:330] T 4c197b05bf284d4c892e3c476cc4cb07 P 220563d55ccc4fac99d9930a81a07316: stopping tablet replica
I20260811 05:13:21.799542 11034 raft_consensus.cc:2258] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:21.799602 11037 raft_consensus.cc:2229] T 4c197b05bf284d4c892e3c476cc4cb07 P 220563d55ccc4fac99d9930a81a07316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:21.799837 11037 raft_consensus.cc:2258] T 4c197b05bf284d4c892e3c476cc4cb07 P 220563d55ccc4fac99d9930a81a07316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:21.800061 10170 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50328:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 05:13:21.800197 11035 tablet_replica.cc:330] T 4c197b05bf284d4c892e3c476cc4cb07 P ade76aa26d02404682b0798c158915aa: stopping tablet replica
W20260811 05:13:21.800459 10170 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:21.800454 11035 raft_consensus.cc:2229] T 4c197b05bf284d4c892e3c476cc4cb07 P ade76aa26d02404682b0798c158915aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:21.800704 11035 raft_consensus.cc:2258] T 4c197b05bf284d4c892e3c476cc4cb07 P ade76aa26d02404682b0798c158915aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:21.801038 11034 ts_tablet_manager.cc:1905] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:21.804090 11034 ts_tablet_manager.cc:1918] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:13:21.804186 11034 log.cc:1196] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/4c197b05bf284d4c892e3c476cc4cb07
I20260811 05:13:21.804469 11034 ts_tablet_manager.cc:1939] T 4c197b05bf284d4c892e3c476cc4cb07 P 1c87338c31514f789c4a801ba48c0595: Deleting consensus metadata
I20260811 05:13:21.804776 11037 ts_tablet_manager.cc:1905] T 4c197b05bf284d4c892e3c476cc4cb07 P 220563d55ccc4fac99d9930a81a07316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:21.805663 10149 catalog_manager.cc:4841] TS 1c87338c31514f789c4a801ba48c0595 (127.7.67.131:38291): tablet 4c197b05bf284d4c892e3c476cc4cb07 (table test-table [id=4abfea41b5104ab4ae533abdfe8f202e]) successfully deleted
I20260811 05:13:21.805716 11035 ts_tablet_manager.cc:1905] T 4c197b05bf284d4c892e3c476cc4cb07 P ade76aa26d02404682b0798c158915aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:21.806126 10818 tablet_service.cc:1467] Processing CreateTablet for tablet b1f3b59b605744b9bfe264d717394053 (DEFAULT_TABLE table=test-table [id=ffb66f12e31843728bffe8da364c9f67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:21.806269 10379 tablet_service.cc:1467] Processing CreateTablet for tablet b1f3b59b605744b9bfe264d717394053 (DEFAULT_TABLE table=test-table [id=ffb66f12e31843728bffe8da364c9f67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:21.806367 10818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1f3b59b605744b9bfe264d717394053. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:21.806383 10595 tablet_service.cc:1467] Processing CreateTablet for tablet b1f3b59b605744b9bfe264d717394053 (DEFAULT_TABLE table=test-table [id=ffb66f12e31843728bffe8da364c9f67]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:21.806854 10379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1f3b59b605744b9bfe264d717394053. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:21.807050 10595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1f3b59b605744b9bfe264d717394053. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:21.808669 10966 tablet_bootstrap.cc:492] T b1f3b59b605744b9bfe264d717394053 P 1c87338c31514f789c4a801ba48c0595: Bootstrap starting.
I20260811 05:13:21.809274 10965 tablet_bootstrap.cc:492] T b1f3b59b605744b9bfe264d717394053 P 220563d55ccc4fac99d9930a81a07316: Bootstrap starting.
I20260811 05:13:21.809787 10966 tablet_bootstrap.cc:654] T b1f3b59b605744b9bfe264d717394053 P 1c87338c31514f789c4a801ba48c0595: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:21.810194 10965 tablet_bootstrap.cc:654] T b1f3b59b605744b9bfe264d717394053 P 220563d55ccc4fac99d9930a81a07316: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:21.811376 10966 tablet_bootstrap.cc:492] T b1f3b59b605744b9bfe264d717394053 P 1c87338c31514f789c4a801ba48c0595: No bootstrap required, opened a new log
I20260811 05:13:21.811460 10966 ts_tablet_manager.cc:1397] T b1f3b59b605744b9bfe264d717394053 P 1c87338c31514f789c4a801ba48c0595: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:13:21.811883 10966 raft_consensus.cc:348] T b1f3b59b605744b9bfe264d717394053 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:21.811980 10966 raft_consensus.cc:374] T b1f3b59b605744b9bfe264d717394053 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:21.812009 10966 raft_consensus.cc:729] T b1f3b59b605744b9bfe264d717394053 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c87338c31514f789c4a801ba48c0595, State: Initialized, Role: FOLLOWER
I20260811 05:13:21.812070 11035 ts_tablet_manager.cc:1918] T 4c197b05bf284d4c892e3c476cc4cb07 P ade76aa26d02404682b0798c158915aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:13:21.812153 11035 log.cc:1196] T 4c197b05bf284d4c892e3c476cc4cb07 P ade76aa26d02404682b0798c158915aa: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/4c197b05bf284d4c892e3c476cc4cb07
I20260811 05:13:21.812119 10966 consensus_queue.cc:260] T b1f3b59b605744b9bfe264d717394053 P 1c87338c31514f789c4a801ba48c0595 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:21.812361 10966 ts_tablet_manager.cc:1428] T b1f3b59b605744b9bfe264d717394053 P 1c87338c31514f789c4a801ba48c0595: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:21.812479 11035 ts_tablet_manager.cc:1939] T 4c197b05bf284d4c892e3c476cc4cb07 P ade76aa26d02404682b0798c158915aa: Deleting consensus metadata
I20260811 05:13:21.813373 10965 tablet_bootstrap.cc:492] T b1f3b59b605744b9bfe264d717394053 P 220563d55ccc4fac99d9930a81a07316: No bootstrap required, opened a new log
I20260811 05:13:21.813452 10965 ts_tablet_manager.cc:1397] T b1f3b59b605744b9bfe264d717394053 P 220563d55ccc4fac99d9930a81a07316: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260811 05:13:21.813898 10965 raft_consensus.cc:348] T b1f3b59b605744b9bfe264d717394053 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:21.814010 10965 raft_consensus.cc:374] T b1f3b59b605744b9bfe264d717394053 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:21.814042 10965 raft_consensus.cc:729] T b1f3b59b605744b9bfe264d717394053 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 220563d55ccc4fac99d9930a81a07316, State: Initialized, Role: FOLLOWER
I20260811 05:13:21.814172 10965 consensus_queue.cc:260] T b1f3b59b605744b9bfe264d717394053 P 220563d55ccc4fac99d9930a81a07316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:21.814453 10965 ts_tablet_manager.cc:1428] T b1f3b59b605744b9bfe264d717394053 P 220563d55ccc4fac99d9930a81a07316: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260811 05:13:21.814872 11037 ts_tablet_manager.cc:1918] T 4c197b05bf284d4c892e3c476cc4cb07 P 220563d55ccc4fac99d9930a81a07316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:13:21.814975 11037 log.cc:1196] T 4c197b05bf284d4c892e3c476cc4cb07 P 220563d55ccc4fac99d9930a81a07316: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-0/wal/wals/4c197b05bf284d4c892e3c476cc4cb07
I20260811 05:13:21.815323 11037 ts_tablet_manager.cc:1939] T 4c197b05bf284d4c892e3c476cc4cb07 P 220563d55ccc4fac99d9930a81a07316: Deleting consensus metadata
I20260811 05:13:21.816560 10967 tablet_bootstrap.cc:492] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa: Bootstrap starting.
I20260811 05:13:21.816764 10151 catalog_manager.cc:4841] TS 220563d55ccc4fac99d9930a81a07316 (127.7.67.129:43129): tablet 4c197b05bf284d4c892e3c476cc4cb07 (table test-table [id=4abfea41b5104ab4ae533abdfe8f202e]) successfully deleted
I20260811 05:13:21.817448 10967 tablet_bootstrap.cc:654] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:21.817554 10152 catalog_manager.cc:4841] TS ade76aa26d02404682b0798c158915aa (127.7.67.130:39631): tablet 4c197b05bf284d4c892e3c476cc4cb07 (table test-table [id=4abfea41b5104ab4ae533abdfe8f202e]) successfully deleted
I20260811 05:13:21.818630 10967 tablet_bootstrap.cc:492] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa: No bootstrap required, opened a new log
I20260811 05:13:21.818722 10967 ts_tablet_manager.cc:1397] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:13:21.819167 10967 raft_consensus.cc:348] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:21.819278 10967 raft_consensus.cc:374] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:21.819301 10967 raft_consensus.cc:729] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ade76aa26d02404682b0798c158915aa, State: Initialized, Role: FOLLOWER
I20260811 05:13:21.819396 10967 consensus_queue.cc:260] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:21.819635 10967 ts_tablet_manager.cc:1428] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 05:13:22.064496 11017 raft_consensus.cc:482] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:22.064776 11017 raft_consensus.cc:504] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:22.065949 11017 leader_election.cc:288] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 220563d55ccc4fac99d9930a81a07316 (127.7.67.129:43129), 1c87338c31514f789c4a801ba48c0595 (127.7.67.131:38291)
I20260811 05:13:22.070714 11044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:22.066255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:38291 (local address 127.7.67.130:33801)
0811 05:13:22.066679 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:22.066711 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:22.066825 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:22.067134 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:22.067138 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:22.067156 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:22.067307 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:22.067315 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:22.068150 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:22.068153 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:22.069309 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:22.069342 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:22.070245 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:22.070249 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:22.070358 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:22.070382 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 05:13:22.070456 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 05:13:22.070533 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":87,"thread_start_us":116,"threads_started":1}
I20260811 05:13:22.070724 10957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:22.066390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:33801 (local address 127.7.67.131:38291)
0811 05:13:22.066532 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:22.066537 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:22.066808 (+ 271us) server_negotiation.cc:407] Connection header received
0811 05:13:22.066957 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:22.066964 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:22.067025 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:22.067103 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:22.067433 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:22.068029 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:22.069504 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:22.070124 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:22.070258 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:22.070308 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:22.070515 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 05:13:22.070595 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 05:13:22.070796 10963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:22.066408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:33865 (local address 127.7.67.129:43129)
0811 05:13:22.066546 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:22.066553 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:22.066902 (+ 349us) server_negotiation.cc:407] Connection header received
0811 05:13:22.067046 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:22.067048 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:22.067134 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:22.067261 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:22.067550 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:22.068051 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:22.069391 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:22.070075 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:22.070232 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:22.070287 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:22.070599 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 05:13:22.070668 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 05:13:22.071148 11048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:22.066255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:43129 (local address 127.7.67.130:33865)
0811 05:13:22.066752 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:22.066783 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:22.066931 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:22.067267 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:22.067269 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:22.067284 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:22.067414 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:22.067420 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:22.068194 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:22.068197 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:22.069261 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:22.069271 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:22.070342 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:22.070345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:22.070442 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:22.070489 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 05:13:22.070930 (+ 441us) client_negotiation.cc:240] Negotiation successful
0811 05:13:22.071004 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"mutex_wait_us":204,"thread_start_us":111,"threads_started":1}
I20260811 05:13:22.071305 10860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1f3b59b605744b9bfe264d717394053" candidate_uuid: "ade76aa26d02404682b0798c158915aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c87338c31514f789c4a801ba48c0595" is_pre_election: true
I20260811 05:13:22.071586 10860 raft_consensus.cc:2454] T b1f3b59b605744b9bfe264d717394053 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ade76aa26d02404682b0798c158915aa in term 0.
I20260811 05:13:22.071581 10406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1f3b59b605744b9bfe264d717394053" candidate_uuid: "ade76aa26d02404682b0798c158915aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "220563d55ccc4fac99d9930a81a07316" is_pre_election: true
I20260811 05:13:22.071743 10406 raft_consensus.cc:2454] T b1f3b59b605744b9bfe264d717394053 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ade76aa26d02404682b0798c158915aa in term 0.
I20260811 05:13:22.072067 10519 leader_election.cc:302] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c87338c31514f789c4a801ba48c0595, ade76aa26d02404682b0798c158915aa; no voters:
I20260811 05:13:22.072361 11017 raft_consensus.cc:2781] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:22.072423 11017 raft_consensus.cc:482] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:22.072463 11017 raft_consensus.cc:3037] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:22.073559 11017 raft_consensus.cc:504] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:22.073899 11017 leader_election.cc:288] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa [CANDIDATE]: Term 1 election: Requested vote from peers 220563d55ccc4fac99d9930a81a07316 (127.7.67.129:43129), 1c87338c31514f789c4a801ba48c0595 (127.7.67.131:38291)
I20260811 05:13:22.074204 10860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1f3b59b605744b9bfe264d717394053" candidate_uuid: "ade76aa26d02404682b0798c158915aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c87338c31514f789c4a801ba48c0595"
I20260811 05:13:22.074208 10406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1f3b59b605744b9bfe264d717394053" candidate_uuid: "ade76aa26d02404682b0798c158915aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "220563d55ccc4fac99d9930a81a07316"
I20260811 05:13:22.074338 10860 raft_consensus.cc:3037] T b1f3b59b605744b9bfe264d717394053 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:22.074355 10406 raft_consensus.cc:3037] T b1f3b59b605744b9bfe264d717394053 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:22.075197 10406 raft_consensus.cc:2454] T b1f3b59b605744b9bfe264d717394053 P 220563d55ccc4fac99d9930a81a07316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ade76aa26d02404682b0798c158915aa in term 1.
I20260811 05:13:22.075320 10860 raft_consensus.cc:2454] T b1f3b59b605744b9bfe264d717394053 P 1c87338c31514f789c4a801ba48c0595 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ade76aa26d02404682b0798c158915aa in term 1.
I20260811 05:13:22.075534 10520 leader_election.cc:302] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 220563d55ccc4fac99d9930a81a07316, ade76aa26d02404682b0798c158915aa; no voters:
I20260811 05:13:22.075726 11017 raft_consensus.cc:2781] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:22.075839 11017 raft_consensus.cc:686] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa [term 1 LEADER]: Becoming Leader. State: Replica: ade76aa26d02404682b0798c158915aa, State: Running, Role: LEADER
I20260811 05:13:22.076079 11017 consensus_queue.cc:237] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:22.078353 10170 catalog_manager.cc:5495] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa reported cstate change: term changed from 0 to 1, leader changed from <none> to ade76aa26d02404682b0798c158915aa (127.7.67.130). New cstate: current_term: 1 leader_uuid: "ade76aa26d02404682b0798c158915aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } health_report { overall_health: UNKNOWN } } }
I20260811 05:13:22.119233 11050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:22.115193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49024 (local address 127.7.67.130:39631)
0811 05:13:22.115472 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:22.115478 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:22.115491 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:13:22.115555 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:22.115557 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:22.115604 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:22.115717 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:22.116280 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:22.116880 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:22.117750 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:22.117982 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:22.118102 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:22.118388 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:22.118833 (+ 445us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:22.118973 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:22.119017 (+ 44us) server_negotiation.cc:299] Negotiation successful
0811 05:13:22.119069 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":133,"threads_started":1}
I20260811 05:13:22.129663 10957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:22.126374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:57013 (local address 127.7.67.131:38291)
0811 05:13:22.126498 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:22.126501 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:22.126716 (+ 215us) server_negotiation.cc:407] Connection header received
0811 05:13:22.126837 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:22.126839 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:22.126882 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:22.126949 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:22.127286 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:22.127761 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:22.128694 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:22.128900 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:22.128980 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:22.129277 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:22.129341 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:22.129401 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:22.129525 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 05:13:22.129565 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 05:13:22.129686 11055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:22.126300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:38291 (local address 127.7.67.130:57013)
0811 05:13:22.126621 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:22.126644 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:22.126741 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:22.126989 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:22.126992 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:22.127001 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:13:22.127128 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:22.127133 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:22.127860 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:22.127863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:22.128574 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:22.128589 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:22.129003 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:22.129006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:22.129083 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:22.129127 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:13:22.129426 (+ 299us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:13:22.129431 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:13:22.129488 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 05:13:22.129552 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":120,"threads_started":1}
I20260811 05:13:22.136101 10406 raft_consensus.cc:1261] T b1f3b59b605744b9bfe264d717394053 P 220563d55ccc4fac99d9930a81a07316 [term 1 FOLLOWER]: Refusing update from remote peer ade76aa26d02404682b0798c158915aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:22.136301 10860 raft_consensus.cc:1261] T b1f3b59b605744b9bfe264d717394053 P 1c87338c31514f789c4a801ba48c0595 [term 1 FOLLOWER]: Refusing update from remote peer ade76aa26d02404682b0798c158915aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:22.136617 11017 consensus_queue.cc:1035] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:22.136864 10975 consensus_queue.cc:1035] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:22.154415 11050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:22.150841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:38417 (local address 127.7.67.130:39631)
0811 05:13:22.150942 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:22.150947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:22.151161 (+ 214us) server_negotiation.cc:407] Connection header received
0811 05:13:22.151292 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:22.151298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:22.151350 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:22.151426 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:22.151648 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:22.152269 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:22.153258 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:22.153559 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:22.153703 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:22.153927 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:22.154016 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:22.154092 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:22.154246 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 05:13:22.154302 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 05:13:22.154474 10933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:22.150794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:39631 (local address 127.7.67.131:38417)
0811 05:13:22.151037 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:22.151050 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:22.151175 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:22.151455 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:22.151460 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:22.151470 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:13:22.151572 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:22.151577 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:22.152366 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:22.152370 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:22.153070 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:22.153080 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:22.153701 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:22.153705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:22.153771 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:22.153800 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:13:22.154112 (+ 312us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:13:22.154119 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 05:13:22.154198 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 05:13:22.154271 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158}
W20260811 05:13:22.155289 11005 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 05:13:23.724799 10171 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:23.724947 10171 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50328:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:23.727428 10171 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115: Sending DeleteTablet for 3 replicas of tablet b1f3b59b605744b9bfe264d717394053
I20260811 05:13:23.727957 10379 tablet_service.cc:1514] Processing DeleteTablet for tablet b1f3b59b605744b9bfe264d717394053 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:23 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:60076
I20260811 05:13:23.728061 10595 tablet_service.cc:1514] Processing DeleteTablet for tablet b1f3b59b605744b9bfe264d717394053 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:23 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:49018
I20260811 05:13:23.728320 11090 tablet_replica.cc:330] T b1f3b59b605744b9bfe264d717394053 P 220563d55ccc4fac99d9930a81a07316: stopping tablet replica
I20260811 05:13:23.728390 10818 tablet_service.cc:1514] Processing DeleteTablet for tablet b1f3b59b605744b9bfe264d717394053 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:23 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:43134
I20260811 05:13:23.728451 11090 raft_consensus.cc:2229] T b1f3b59b605744b9bfe264d717394053 P 220563d55ccc4fac99d9930a81a07316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:23.728595 11090 raft_consensus.cc:2258] T b1f3b59b605744b9bfe264d717394053 P 220563d55ccc4fac99d9930a81a07316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:23.728765 11091 tablet_replica.cc:330] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa: stopping tablet replica
I20260811 05:13:23.728871 11092 tablet_replica.cc:330] T b1f3b59b605744b9bfe264d717394053 P 1c87338c31514f789c4a801ba48c0595: stopping tablet replica
I20260811 05:13:23.728896 11091 raft_consensus.cc:2229] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:13:23.729089 11092 raft_consensus.cc:2229] T b1f3b59b605744b9bfe264d717394053 P 1c87338c31514f789c4a801ba48c0595 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:23.729195 11091 raft_consensus.cc:2258] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:23.729245 11090 ts_tablet_manager.cc:1905] T b1f3b59b605744b9bfe264d717394053 P 220563d55ccc4fac99d9930a81a07316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:23.729333 11092 raft_consensus.cc:2258] T b1f3b59b605744b9bfe264d717394053 P 1c87338c31514f789c4a801ba48c0595 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:23.730059 11092 ts_tablet_manager.cc:1905] T b1f3b59b605744b9bfe264d717394053 P 1c87338c31514f789c4a801ba48c0595: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:23.732463 11090 ts_tablet_manager.cc:1918] T b1f3b59b605744b9bfe264d717394053 P 220563d55ccc4fac99d9930a81a07316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 05:13:23.732599 11090 log.cc:1196] T b1f3b59b605744b9bfe264d717394053 P 220563d55ccc4fac99d9930a81a07316: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-0/wal/wals/b1f3b59b605744b9bfe264d717394053
I20260811 05:13:23.732590 11091 ts_tablet_manager.cc:1905] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:23.732856 11092 ts_tablet_manager.cc:1918] T b1f3b59b605744b9bfe264d717394053 P 1c87338c31514f789c4a801ba48c0595: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 05:13:23.732950 11092 log.cc:1196] T b1f3b59b605744b9bfe264d717394053 P 1c87338c31514f789c4a801ba48c0595: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/b1f3b59b605744b9bfe264d717394053
I20260811 05:13:23.732966 11090 ts_tablet_manager.cc:1939] T b1f3b59b605744b9bfe264d717394053 P 220563d55ccc4fac99d9930a81a07316: Deleting consensus metadata
I20260811 05:13:23.733294 11092 ts_tablet_manager.cc:1939] T b1f3b59b605744b9bfe264d717394053 P 1c87338c31514f789c4a801ba48c0595: Deleting consensus metadata
I20260811 05:13:23.733732 10151 catalog_manager.cc:4841] TS 220563d55ccc4fac99d9930a81a07316 (127.7.67.129:43129): tablet b1f3b59b605744b9bfe264d717394053 (table test-table [id=ffb66f12e31843728bffe8da364c9f67]) successfully deleted
I20260811 05:13:23.733979 10149 catalog_manager.cc:4841] TS 1c87338c31514f789c4a801ba48c0595 (127.7.67.131:38291): tablet b1f3b59b605744b9bfe264d717394053 (table test-table [id=ffb66f12e31843728bffe8da364c9f67]) successfully deleted
I20260811 05:13:23.737416 11091 ts_tablet_manager.cc:1918] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 05:13:23.737505 11091 log.cc:1196] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/b1f3b59b605744b9bfe264d717394053
I20260811 05:13:23.737818 11091 ts_tablet_manager.cc:1939] T b1f3b59b605744b9bfe264d717394053 P ade76aa26d02404682b0798c158915aa: Deleting consensus metadata
I20260811 05:13:23.738555 10152 catalog_manager.cc:4841] TS ade76aa26d02404682b0798c158915aa (127.7.67.130:39631): tablet b1f3b59b605744b9bfe264d717394053 (table test-table [id=ffb66f12e31843728bffe8da364c9f67]) successfully deleted
Aug 11 05:13:23 dist-test-slave-04bb krb5kdc[10103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425203, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 05:13:23 dist-test-slave-04bb krb5kdc[10103](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425203, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:23.760963 11102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:23.752069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50334 (local address 127.7.67.190:35425)
0811 05:13:23.752520 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:23.752567 (+ 47us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:23.752582 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:13:23.752628 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:23.752630 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:23.752675 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:23.752756 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:23.753455 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:23.754004 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:23.754842 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:23.755067 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:23.755145 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:23.757696 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:23.757700 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:23.757702 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:23.757725 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:23.759380 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:23.759763 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:23.759766 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:23.759767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:23.759868 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:23.760139 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:23.760142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:23.760143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:23.760373 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:23.760494 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:23.760737 (+ 243us) server_negotiation.cc:299] Negotiation successful
0811 05:13:23.760840 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":130,"threads_started":1}
I20260811 05:13:23.763574 10171 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:13:23.764060 10171 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:23.769018 10379 tablet_service.cc:1467] Processing CreateTablet for tablet a4ed905ff9444682936b4229c26dbc38 (DEFAULT_TABLE table=test-table [id=a43067b977d04fae9f5a8e3867c87aa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:23.769304 10818 tablet_service.cc:1467] Processing CreateTablet for tablet a4ed905ff9444682936b4229c26dbc38 (DEFAULT_TABLE table=test-table [id=a43067b977d04fae9f5a8e3867c87aa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:23.769335 10595 tablet_service.cc:1467] Processing CreateTablet for tablet a4ed905ff9444682936b4229c26dbc38 (DEFAULT_TABLE table=test-table [id=a43067b977d04fae9f5a8e3867c87aa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:23.769423 10379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4ed905ff9444682936b4229c26dbc38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:23.769593 10818 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4ed905ff9444682936b4229c26dbc38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:23.769663 10595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4ed905ff9444682936b4229c26dbc38. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:23.771880 11103 tablet_bootstrap.cc:492] T a4ed905ff9444682936b4229c26dbc38 P ade76aa26d02404682b0798c158915aa: Bootstrap starting.
I20260811 05:13:23.772364 11105 tablet_bootstrap.cc:492] T a4ed905ff9444682936b4229c26dbc38 P 220563d55ccc4fac99d9930a81a07316: Bootstrap starting.
I20260811 05:13:23.772787 11103 tablet_bootstrap.cc:654] T a4ed905ff9444682936b4229c26dbc38 P ade76aa26d02404682b0798c158915aa: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:23.773303 11105 tablet_bootstrap.cc:654] T a4ed905ff9444682936b4229c26dbc38 P 220563d55ccc4fac99d9930a81a07316: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:23.774201 11103 tablet_bootstrap.cc:492] T a4ed905ff9444682936b4229c26dbc38 P ade76aa26d02404682b0798c158915aa: No bootstrap required, opened a new log
I20260811 05:13:23.774298 11103 ts_tablet_manager.cc:1397] T a4ed905ff9444682936b4229c26dbc38 P ade76aa26d02404682b0798c158915aa: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:13:23.774420 11104 tablet_bootstrap.cc:492] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595: Bootstrap starting.
I20260811 05:13:23.774792 11103 raft_consensus.cc:348] T a4ed905ff9444682936b4229c26dbc38 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:23.774914 11103 raft_consensus.cc:374] T a4ed905ff9444682936b4229c26dbc38 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:23.774946 11103 raft_consensus.cc:729] T a4ed905ff9444682936b4229c26dbc38 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ade76aa26d02404682b0798c158915aa, State: Initialized, Role: FOLLOWER
I20260811 05:13:23.775103 11103 consensus_queue.cc:260] T a4ed905ff9444682936b4229c26dbc38 P ade76aa26d02404682b0798c158915aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:23.775383 11104 tablet_bootstrap.cc:654] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:23.775425 11103 ts_tablet_manager.cc:1428] T a4ed905ff9444682936b4229c26dbc38 P ade76aa26d02404682b0798c158915aa: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 05:13:23.776371 11105 tablet_bootstrap.cc:492] T a4ed905ff9444682936b4229c26dbc38 P 220563d55ccc4fac99d9930a81a07316: No bootstrap required, opened a new log
I20260811 05:13:23.776484 11105 ts_tablet_manager.cc:1397] T a4ed905ff9444682936b4229c26dbc38 P 220563d55ccc4fac99d9930a81a07316: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 05:13:23.776995 11104 tablet_bootstrap.cc:492] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595: No bootstrap required, opened a new log
I20260811 05:13:23.777001 11105 raft_consensus.cc:348] T a4ed905ff9444682936b4229c26dbc38 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:23.777092 11104 ts_tablet_manager.cc:1397] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:13:23.777119 11105 raft_consensus.cc:374] T a4ed905ff9444682936b4229c26dbc38 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:23.777176 11105 raft_consensus.cc:729] T a4ed905ff9444682936b4229c26dbc38 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 220563d55ccc4fac99d9930a81a07316, State: Initialized, Role: FOLLOWER
I20260811 05:13:23.777304 11105 consensus_queue.cc:260] T a4ed905ff9444682936b4229c26dbc38 P 220563d55ccc4fac99d9930a81a07316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:23.777575 11104 raft_consensus.cc:348] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:23.777684 11104 raft_consensus.cc:374] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:23.777697 11105 ts_tablet_manager.cc:1428] T a4ed905ff9444682936b4229c26dbc38 P 220563d55ccc4fac99d9930a81a07316: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 05:13:23.777719 11104 raft_consensus.cc:729] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c87338c31514f789c4a801ba48c0595, State: Initialized, Role: FOLLOWER
I20260811 05:13:23.777832 11104 consensus_queue.cc:260] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:23.778101 11104 ts_tablet_manager.cc:1428] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 05:13:24.088661 11065 raft_consensus.cc:482] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:24.088802 11065 raft_consensus.cc:504] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:24.089310 11065 leader_election.cc:288] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 220563d55ccc4fac99d9930a81a07316 (127.7.67.129:43129), ade76aa26d02404682b0798c158915aa (127.7.67.130:39631)
I20260811 05:13:24.089789 10627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ed905ff9444682936b4229c26dbc38" candidate_uuid: "1c87338c31514f789c4a801ba48c0595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ade76aa26d02404682b0798c158915aa" is_pre_election: true
I20260811 05:13:24.089807 10406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ed905ff9444682936b4229c26dbc38" candidate_uuid: "1c87338c31514f789c4a801ba48c0595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "220563d55ccc4fac99d9930a81a07316" is_pre_election: true
I20260811 05:13:24.089957 10627 raft_consensus.cc:2454] T a4ed905ff9444682936b4229c26dbc38 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c87338c31514f789c4a801ba48c0595 in term 0.
I20260811 05:13:24.089972 10406 raft_consensus.cc:2454] T a4ed905ff9444682936b4229c26dbc38 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c87338c31514f789c4a801ba48c0595 in term 0.
I20260811 05:13:24.090317 10745 leader_election.cc:302] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c87338c31514f789c4a801ba48c0595, ade76aa26d02404682b0798c158915aa; no voters:
I20260811 05:13:24.090502 11065 raft_consensus.cc:2781] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:24.090579 11065 raft_consensus.cc:482] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:24.090636 11065 raft_consensus.cc:3037] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:24.091945 11065 raft_consensus.cc:504] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:24.092386 11065 leader_election.cc:288] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595 [CANDIDATE]: Term 1 election: Requested vote from peers 220563d55ccc4fac99d9930a81a07316 (127.7.67.129:43129), ade76aa26d02404682b0798c158915aa (127.7.67.130:39631)
I20260811 05:13:24.092988 10627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ed905ff9444682936b4229c26dbc38" candidate_uuid: "1c87338c31514f789c4a801ba48c0595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ade76aa26d02404682b0798c158915aa"
I20260811 05:13:24.093111 10627 raft_consensus.cc:3037] T a4ed905ff9444682936b4229c26dbc38 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:24.094244 10627 raft_consensus.cc:2454] T a4ed905ff9444682936b4229c26dbc38 P ade76aa26d02404682b0798c158915aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c87338c31514f789c4a801ba48c0595 in term 1.
I20260811 05:13:24.094504 10406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ed905ff9444682936b4229c26dbc38" candidate_uuid: "1c87338c31514f789c4a801ba48c0595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "220563d55ccc4fac99d9930a81a07316"
I20260811 05:13:24.094549 10745 leader_election.cc:302] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c87338c31514f789c4a801ba48c0595, ade76aa26d02404682b0798c158915aa; no voters:
I20260811 05:13:24.094631 10406 raft_consensus.cc:3037] T a4ed905ff9444682936b4229c26dbc38 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:24.094962 11065 raft_consensus.cc:2781] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:24.095634 11065 raft_consensus.cc:686] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595 [term 1 LEADER]: Becoming Leader. State: Replica: 1c87338c31514f789c4a801ba48c0595, State: Running, Role: LEADER
I20260811 05:13:24.095662 10406 raft_consensus.cc:2454] T a4ed905ff9444682936b4229c26dbc38 P 220563d55ccc4fac99d9930a81a07316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c87338c31514f789c4a801ba48c0595 in term 1.
I20260811 05:13:24.095983 11065 consensus_queue.cc:237] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:24.098315 10171 catalog_manager.cc:5495] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c87338c31514f789c4a801ba48c0595 (127.7.67.131). New cstate: current_term: 1 leader_uuid: "1c87338c31514f789c4a801ba48c0595" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } health_report { overall_health: HEALTHY } } }
I20260811 05:13:24.145300 11118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:24.142099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43168 (local address 127.7.67.131:38291)
0811 05:13:24.142406 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:24.142414 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:24.142426 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:13:24.142465 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:24.142468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:24.142510 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:24.142585 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:24.142857 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.143387 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.144184 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:24.144417 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:24.144502 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:24.144815 (+ 313us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:24.144903 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:24.144965 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:24.145052 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 05:13:24.145101 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":137,"threads_started":1}
I20260811 05:13:24.151492 10406 raft_consensus.cc:1261] T a4ed905ff9444682936b4229c26dbc38 P 220563d55ccc4fac99d9930a81a07316 [term 1 FOLLOWER]: Refusing update from remote peer 1c87338c31514f789c4a801ba48c0595: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:24.151857 10627 raft_consensus.cc:1261] T a4ed905ff9444682936b4229c26dbc38 P ade76aa26d02404682b0798c158915aa [term 1 FOLLOWER]: Refusing update from remote peer 1c87338c31514f789c4a801ba48c0595: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:24.152262 11116 consensus_queue.cc:1035] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:24.152412 11065 consensus_queue.cc:1035] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595 [LEADER]: Connected to new peer: Peer: permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:24.184516 10170 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:24.184731 10170 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:24.187050 10170 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115: Sending DeleteTablet for 3 replicas of tablet a4ed905ff9444682936b4229c26dbc38
I20260811 05:13:24.187695 10379 tablet_service.cc:1514] Processing DeleteTablet for tablet a4ed905ff9444682936b4229c26dbc38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:24 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:60076
I20260811 05:13:24.187799 10595 tablet_service.cc:1514] Processing DeleteTablet for tablet a4ed905ff9444682936b4229c26dbc38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:24 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:49018
I20260811 05:13:24.188041 11090 tablet_replica.cc:330] T a4ed905ff9444682936b4229c26dbc38 P 220563d55ccc4fac99d9930a81a07316: stopping tablet replica
I20260811 05:13:24.188231 11090 raft_consensus.cc:2229] T a4ed905ff9444682936b4229c26dbc38 P 220563d55ccc4fac99d9930a81a07316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:24.188381 11090 raft_consensus.cc:2258] T a4ed905ff9444682936b4229c26dbc38 P 220563d55ccc4fac99d9930a81a07316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:24.188604 11091 tablet_replica.cc:330] T a4ed905ff9444682936b4229c26dbc38 P ade76aa26d02404682b0798c158915aa: stopping tablet replica
I20260811 05:13:24.188714 10819 tablet_service.cc:1514] Processing DeleteTablet for tablet a4ed905ff9444682936b4229c26dbc38 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:24 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:43134
I20260811 05:13:24.188751 11091 raft_consensus.cc:2229] T a4ed905ff9444682936b4229c26dbc38 P ade76aa26d02404682b0798c158915aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:24.188879 11091 raft_consensus.cc:2258] T a4ed905ff9444682936b4229c26dbc38 P ade76aa26d02404682b0798c158915aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:24.188930 11092 tablet_replica.cc:330] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595: stopping tablet replica
I20260811 05:13:24.189055 11092 raft_consensus.cc:2229] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:13:24.189239 11092 raft_consensus.cc:2258] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:24.189281 10170 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50334:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 05:13:24.189899 11090 ts_tablet_manager.cc:1905] T a4ed905ff9444682936b4229c26dbc38 P 220563d55ccc4fac99d9930a81a07316: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:24.189916 11092 ts_tablet_manager.cc:1905] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:24.189913 11091 ts_tablet_manager.cc:1905] T a4ed905ff9444682936b4229c26dbc38 P ade76aa26d02404682b0798c158915aa: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 05:13:24.190061 10170 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:24.192498 11092 ts_tablet_manager.cc:1918] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:13:24.192605 11092 log.cc:1196] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/a4ed905ff9444682936b4229c26dbc38
I20260811 05:13:24.192785 11090 ts_tablet_manager.cc:1918] T a4ed905ff9444682936b4229c26dbc38 P 220563d55ccc4fac99d9930a81a07316: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:13:24.192876 11090 log.cc:1196] T a4ed905ff9444682936b4229c26dbc38 P 220563d55ccc4fac99d9930a81a07316: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-0/wal/wals/a4ed905ff9444682936b4229c26dbc38
I20260811 05:13:24.192883 11092 ts_tablet_manager.cc:1939] T a4ed905ff9444682936b4229c26dbc38 P 1c87338c31514f789c4a801ba48c0595: Deleting consensus metadata
I20260811 05:13:24.193157 11090 ts_tablet_manager.cc:1939] T a4ed905ff9444682936b4229c26dbc38 P 220563d55ccc4fac99d9930a81a07316: Deleting consensus metadata
I20260811 05:13:24.193224 11091 ts_tablet_manager.cc:1918] T a4ed905ff9444682936b4229c26dbc38 P ade76aa26d02404682b0798c158915aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:13:24.193320 11091 log.cc:1196] T a4ed905ff9444682936b4229c26dbc38 P ade76aa26d02404682b0798c158915aa: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/a4ed905ff9444682936b4229c26dbc38
I20260811 05:13:24.193588 11091 ts_tablet_manager.cc:1939] T a4ed905ff9444682936b4229c26dbc38 P ade76aa26d02404682b0798c158915aa: Deleting consensus metadata
I20260811 05:13:24.193928 10151 catalog_manager.cc:4841] TS 220563d55ccc4fac99d9930a81a07316 (127.7.67.129:43129): tablet a4ed905ff9444682936b4229c26dbc38 (table test-table [id=a43067b977d04fae9f5a8e3867c87aa5]) successfully deleted
I20260811 05:13:24.194048 10149 catalog_manager.cc:4841] TS 1c87338c31514f789c4a801ba48c0595 (127.7.67.131:38291): tablet a4ed905ff9444682936b4229c26dbc38 (table test-table [id=a43067b977d04fae9f5a8e3867c87aa5]) successfully deleted
I20260811 05:13:24.194324 10152 catalog_manager.cc:4841] TS ade76aa26d02404682b0798c158915aa (127.7.67.130:39631): tablet a4ed905ff9444682936b4229c26dbc38 (table test-table [id=a43067b977d04fae9f5a8e3867c87aa5]) successfully deleted
I20260811 05:13:24.196362 10379 tablet_service.cc:1467] Processing CreateTablet for tablet c9caeaa25ac94515b6c56cf4b8a62674 (DEFAULT_TABLE table=test-table [id=84f6f33b50724a7384c852d3d44824f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:24.196494 10595 tablet_service.cc:1467] Processing CreateTablet for tablet c9caeaa25ac94515b6c56cf4b8a62674 (DEFAULT_TABLE table=test-table [id=84f6f33b50724a7384c852d3d44824f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:24.196691 10379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9caeaa25ac94515b6c56cf4b8a62674. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:24.196677 10819 tablet_service.cc:1467] Processing CreateTablet for tablet c9caeaa25ac94515b6c56cf4b8a62674 (DEFAULT_TABLE table=test-table [id=84f6f33b50724a7384c852d3d44824f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:24.196779 10595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9caeaa25ac94515b6c56cf4b8a62674. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:24.196938 10819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c9caeaa25ac94515b6c56cf4b8a62674. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:24.198629 11105 tablet_bootstrap.cc:492] T c9caeaa25ac94515b6c56cf4b8a62674 P 220563d55ccc4fac99d9930a81a07316: Bootstrap starting.
I20260811 05:13:24.199079 11103 tablet_bootstrap.cc:492] T c9caeaa25ac94515b6c56cf4b8a62674 P ade76aa26d02404682b0798c158915aa: Bootstrap starting.
I20260811 05:13:24.199414 11104 tablet_bootstrap.cc:492] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595: Bootstrap starting.
I20260811 05:13:24.199641 11105 tablet_bootstrap.cc:654] T c9caeaa25ac94515b6c56cf4b8a62674 P 220563d55ccc4fac99d9930a81a07316: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:24.199990 11103 tablet_bootstrap.cc:654] T c9caeaa25ac94515b6c56cf4b8a62674 P ade76aa26d02404682b0798c158915aa: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:24.200048 11104 tablet_bootstrap.cc:654] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:24.201133 11105 tablet_bootstrap.cc:492] T c9caeaa25ac94515b6c56cf4b8a62674 P 220563d55ccc4fac99d9930a81a07316: No bootstrap required, opened a new log
I20260811 05:13:24.201244 11104 tablet_bootstrap.cc:492] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595: No bootstrap required, opened a new log
I20260811 05:13:24.201254 11105 ts_tablet_manager.cc:1397] T c9caeaa25ac94515b6c56cf4b8a62674 P 220563d55ccc4fac99d9930a81a07316: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260811 05:13:24.201355 11104 ts_tablet_manager.cc:1397] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:24.201678 11105 raft_consensus.cc:348] T c9caeaa25ac94515b6c56cf4b8a62674 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:24.201711 11104 raft_consensus.cc:348] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:24.201807 11105 raft_consensus.cc:374] T c9caeaa25ac94515b6c56cf4b8a62674 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:24.201803 11104 raft_consensus.cc:374] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:24.201853 11104 raft_consensus.cc:729] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c87338c31514f789c4a801ba48c0595, State: Initialized, Role: FOLLOWER
I20260811 05:13:24.201853 11105 raft_consensus.cc:729] T c9caeaa25ac94515b6c56cf4b8a62674 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 220563d55ccc4fac99d9930a81a07316, State: Initialized, Role: FOLLOWER
I20260811 05:13:24.201985 11104 consensus_queue.cc:260] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:24.201992 11105 consensus_queue.cc:260] T c9caeaa25ac94515b6c56cf4b8a62674 P 220563d55ccc4fac99d9930a81a07316 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:24.202284 11104 ts_tablet_manager.cc:1428] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 05:13:24.202445 11105 ts_tablet_manager.cc:1428] T c9caeaa25ac94515b6c56cf4b8a62674 P 220563d55ccc4fac99d9930a81a07316: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260811 05:13:24.203306 11103 tablet_bootstrap.cc:492] T c9caeaa25ac94515b6c56cf4b8a62674 P ade76aa26d02404682b0798c158915aa: No bootstrap required, opened a new log
I20260811 05:13:24.203428 11103 ts_tablet_manager.cc:1397] T c9caeaa25ac94515b6c56cf4b8a62674 P ade76aa26d02404682b0798c158915aa: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260811 05:13:24.203902 11103 raft_consensus.cc:348] T c9caeaa25ac94515b6c56cf4b8a62674 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:24.204046 11103 raft_consensus.cc:374] T c9caeaa25ac94515b6c56cf4b8a62674 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:24.204089 11103 raft_consensus.cc:729] T c9caeaa25ac94515b6c56cf4b8a62674 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ade76aa26d02404682b0798c158915aa, State: Initialized, Role: FOLLOWER
I20260811 05:13:24.204221 11103 consensus_queue.cc:260] T c9caeaa25ac94515b6c56cf4b8a62674 P ade76aa26d02404682b0798c158915aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:24.204497 11103 ts_tablet_manager.cc:1428] T c9caeaa25ac94515b6c56cf4b8a62674 P ade76aa26d02404682b0798c158915aa: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:24.297668 11065 raft_consensus.cc:482] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:24.297811 11065 raft_consensus.cc:504] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:24.298298 11065 leader_election.cc:288] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 220563d55ccc4fac99d9930a81a07316 (127.7.67.129:43129), ade76aa26d02404682b0798c158915aa (127.7.67.130:39631)
I20260811 05:13:24.298753 10627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9caeaa25ac94515b6c56cf4b8a62674" candidate_uuid: "1c87338c31514f789c4a801ba48c0595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ade76aa26d02404682b0798c158915aa" is_pre_election: true
I20260811 05:13:24.298785 10406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9caeaa25ac94515b6c56cf4b8a62674" candidate_uuid: "1c87338c31514f789c4a801ba48c0595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "220563d55ccc4fac99d9930a81a07316" is_pre_election: true
I20260811 05:13:24.298923 10627 raft_consensus.cc:2454] T c9caeaa25ac94515b6c56cf4b8a62674 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c87338c31514f789c4a801ba48c0595 in term 0.
I20260811 05:13:24.298974 10406 raft_consensus.cc:2454] T c9caeaa25ac94515b6c56cf4b8a62674 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1c87338c31514f789c4a801ba48c0595 in term 0.
I20260811 05:13:24.299280 10745 leader_election.cc:302] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c87338c31514f789c4a801ba48c0595, ade76aa26d02404682b0798c158915aa; no voters:
I20260811 05:13:24.299458 11065 raft_consensus.cc:2781] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:24.299517 11065 raft_consensus.cc:482] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:24.299546 11065 raft_consensus.cc:3037] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:24.300550 11065 raft_consensus.cc:504] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:24.300860 11065 leader_election.cc:288] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595 [CANDIDATE]: Term 1 election: Requested vote from peers 220563d55ccc4fac99d9930a81a07316 (127.7.67.129:43129), ade76aa26d02404682b0798c158915aa (127.7.67.130:39631)
I20260811 05:13:24.301198 10406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9caeaa25ac94515b6c56cf4b8a62674" candidate_uuid: "1c87338c31514f789c4a801ba48c0595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "220563d55ccc4fac99d9930a81a07316"
I20260811 05:13:24.301231 10627 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c9caeaa25ac94515b6c56cf4b8a62674" candidate_uuid: "1c87338c31514f789c4a801ba48c0595" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ade76aa26d02404682b0798c158915aa"
I20260811 05:13:24.301322 10406 raft_consensus.cc:3037] T c9caeaa25ac94515b6c56cf4b8a62674 P 220563d55ccc4fac99d9930a81a07316 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:24.301331 10627 raft_consensus.cc:3037] T c9caeaa25ac94515b6c56cf4b8a62674 P ade76aa26d02404682b0798c158915aa [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:24.302440 10406 raft_consensus.cc:2454] T c9caeaa25ac94515b6c56cf4b8a62674 P 220563d55ccc4fac99d9930a81a07316 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c87338c31514f789c4a801ba48c0595 in term 1.
I20260811 05:13:24.302440 10627 raft_consensus.cc:2454] T c9caeaa25ac94515b6c56cf4b8a62674 P ade76aa26d02404682b0798c158915aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1c87338c31514f789c4a801ba48c0595 in term 1.
I20260811 05:13:24.302747 10744 leader_election.cc:302] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1c87338c31514f789c4a801ba48c0595, 220563d55ccc4fac99d9930a81a07316; no voters:
I20260811 05:13:24.302965 11065 raft_consensus.cc:2781] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:24.303083 11065 raft_consensus.cc:686] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595 [term 1 LEADER]: Becoming Leader. State: Replica: 1c87338c31514f789c4a801ba48c0595, State: Running, Role: LEADER
I20260811 05:13:24.303229 11065 consensus_queue.cc:237] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } }
I20260811 05:13:24.305084 10170 catalog_manager.cc:5495] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1c87338c31514f789c4a801ba48c0595 (127.7.67.131). New cstate: current_term: 1 leader_uuid: "1c87338c31514f789c4a801ba48c0595" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1c87338c31514f789c4a801ba48c0595" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38291 } health_report { overall_health: HEALTHY } } }
I20260811 05:13:24.335482 10627 raft_consensus.cc:1261] T c9caeaa25ac94515b6c56cf4b8a62674 P ade76aa26d02404682b0798c158915aa [term 1 FOLLOWER]: Refusing update from remote peer 1c87338c31514f789c4a801ba48c0595: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:24.335862 10406 raft_consensus.cc:1261] T c9caeaa25ac94515b6c56cf4b8a62674 P 220563d55ccc4fac99d9930a81a07316 [term 1 FOLLOWER]: Refusing update from remote peer 1c87338c31514f789c4a801ba48c0595: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:24.336033 11116 consensus_queue.cc:1035] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ade76aa26d02404682b0798c158915aa" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:24.336242 11065 consensus_queue.cc:1035] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595 [LEADER]: Connected to new peer: Peer: permanent_uuid: "220563d55ccc4fac99d9930a81a07316" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 05:13:24.350569 11063 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 05:13:25.903147 10170 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:25.903319 10170 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50334:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:25.905655 10170 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 688908b3559d42219236a70e7cb58115: Sending DeleteTablet for 3 replicas of tablet c9caeaa25ac94515b6c56cf4b8a62674
I20260811 05:13:25.906288 10379 tablet_service.cc:1514] Processing DeleteTablet for tablet c9caeaa25ac94515b6c56cf4b8a62674 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:25 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:60076
I20260811 05:13:25.906394 10595 tablet_service.cc:1514] Processing DeleteTablet for tablet c9caeaa25ac94515b6c56cf4b8a62674 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:25 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:49018
I20260811 05:13:25.906426 10818 tablet_service.cc:1514] Processing DeleteTablet for tablet c9caeaa25ac94515b6c56cf4b8a62674 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:25 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:43134
I20260811 05:13:25.906702 11166 tablet_replica.cc:330] T c9caeaa25ac94515b6c56cf4b8a62674 P 220563d55ccc4fac99d9930a81a07316: stopping tablet replica
I20260811 05:13:25.906766 11167 tablet_replica.cc:330] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595: stopping tablet replica
I20260811 05:13:25.906860 11166 raft_consensus.cc:2229] T c9caeaa25ac94515b6c56cf4b8a62674 P 220563d55ccc4fac99d9930a81a07316 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:25.906870 11167 raft_consensus.cc:2229] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:13:25.907040 11166 raft_consensus.cc:2258] T c9caeaa25ac94515b6c56cf4b8a62674 P 220563d55ccc4fac99d9930a81a07316 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:25.907148 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 10281
I20260811 05:13:25.919301 11169 tablet_replica.cc:330] T c9caeaa25ac94515b6c56cf4b8a62674 P ade76aa26d02404682b0798c158915aa: stopping tablet replica
I20260811 05:13:25.919541 11169 raft_consensus.cc:2229] T c9caeaa25ac94515b6c56cf4b8a62674 P ade76aa26d02404682b0798c158915aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:25.919719 11169 raft_consensus.cc:2258] T c9caeaa25ac94515b6c56cf4b8a62674 P ade76aa26d02404682b0798c158915aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:25.920047 11167 raft_consensus.cc:2258] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:25.920755 11169 ts_tablet_manager.cc:1905] T c9caeaa25ac94515b6c56cf4b8a62674 P ade76aa26d02404682b0798c158915aa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:25.923209 11169 ts_tablet_manager.cc:1918] T c9caeaa25ac94515b6c56cf4b8a62674 P ade76aa26d02404682b0798c158915aa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 05:13:25.923317 11169 log.cc:1196] T c9caeaa25ac94515b6c56cf4b8a62674 P ade76aa26d02404682b0798c158915aa: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/c9caeaa25ac94515b6c56cf4b8a62674
I20260811 05:13:25.923564 11169 ts_tablet_manager.cc:1939] T c9caeaa25ac94515b6c56cf4b8a62674 P ade76aa26d02404682b0798c158915aa: Deleting consensus metadata
I20260811 05:13:25.924400 10152 catalog_manager.cc:4841] TS ade76aa26d02404682b0798c158915aa (127.7.67.130:39631): tablet c9caeaa25ac94515b6c56cf4b8a62674 (table test-table [id=84f6f33b50724a7384c852d3d44824f7]) successfully deleted
I20260811 05:13:25.924674 11167 ts_tablet_manager.cc:1905] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:25.926927 11167 ts_tablet_manager.cc:1918] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 05:13:25.927023 11167 log.cc:1196] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/c9caeaa25ac94515b6c56cf4b8a62674
I20260811 05:13:25.927336 11167 ts_tablet_manager.cc:1939] T c9caeaa25ac94515b6c56cf4b8a62674 P 1c87338c31514f789c4a801ba48c0595: Deleting consensus metadata
I20260811 05:13:25.928153 10149 catalog_manager.cc:4841] TS 1c87338c31514f789c4a801ba48c0595 (127.7.67.131:38291): tablet c9caeaa25ac94515b6c56cf4b8a62674 (table test-table [id=84f6f33b50724a7384c852d3d44824f7]) successfully deleted
W20260811 05:13:25.936053 10151 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.129:43129 (error 108)
I20260811 05:13:25.936347 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 10496
I20260811 05:13:25.941099 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:25.938966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:43129 (local address 127.0.0.1:54818)
0811 05:13:25.940780 (+ 1814us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:25.940931 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.129:43129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1694,"thread_start_us":948,"threads_started":1}
W20260811 05:13:25.941260 10151 catalog_manager.cc:4584] TS 220563d55ccc4fac99d9930a81a07316 (127.7.67.129:43129): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c9caeaa25ac94515b6c56cf4b8a62674: Network error: Client connection negotiation failed: client connection to 127.7.67.129:43129: connect: Connection refused (error 111)
I20260811 05:13:25.951360 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 10719
I20260811 05:13:25.969197 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 10124
2026-08-11T05:13:25Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (7043 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:13:26 dist-test-slave-04bb krb5kdc[11177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:13:26 dist-test-slave-04bb krb5kdc[11177](info): set up 2 sockets
Aug 11 05:13:26 dist-test-slave-04bb krb5kdc[11177](info): commencing operation
krb5kdc: starting...
W20260811 05:13:27.248440 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:13:27 dist-test-slave-04bb krb5kdc[11177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:13:27.376412 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:34077
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:46157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:34077
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:27.490511 11195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:27.490772 11195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:27.490813 11195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:27.494901 11195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:27.494963 11195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:27.494983 11195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:27.495000 11195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:27.495015 11195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:27.500279 11195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:46157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:34077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:34077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:27.501386 11195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:27.502453 11195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:27.511157 11206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:27.511195 11203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:27.511170 11208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:27.511405 11195 server_base.cc:1034] running on GCE node
I20260811 05:13:27.512059 11195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:27.513221 11195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:27.514415 11195 hybrid_clock.cc:648] HybridClock initialized: now 1786425207514385 us; error 44 us; skew 500 ppm
Aug 11 05:13:27 dist-test-slave-04bb krb5kdc[11177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425207, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:27.517194 11195 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:13:27.518111 11195 webserver.cc:460] Webserver started at http://127.7.67.190:39417/ using document root <none> and password file <none>
I20260811 05:13:27.518632 11195 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:27.518689 11195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:27.518864 11195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:27.520922 11195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "eac7edf5a2db4e1f9e8843bfa72d6fb6"
format_stamp: "Formatted at 2026-08-11 05:13:27 on dist-test-slave-04bb"
server_key: "7ac4f4119dcb7856018a2efe056c6db0"
server_key_iv: "4424589e07d2bdf502e954258dacface"
server_key_version: "encryptionkey@0"
I20260811 05:13:27.521436 11195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "eac7edf5a2db4e1f9e8843bfa72d6fb6"
format_stamp: "Formatted at 2026-08-11 05:13:27 on dist-test-slave-04bb"
server_key: "7ac4f4119dcb7856018a2efe056c6db0"
server_key_iv: "4424589e07d2bdf502e954258dacface"
server_key_version: "encryptionkey@0"
I20260811 05:13:27.525290 11195 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 05:13:27.527704 11219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:27.529047 11195 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:13:27.529146 11195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "eac7edf5a2db4e1f9e8843bfa72d6fb6"
format_stamp: "Formatted at 2026-08-11 05:13:27 on dist-test-slave-04bb"
server_key: "7ac4f4119dcb7856018a2efe056c6db0"
server_key_iv: "4424589e07d2bdf502e954258dacface"
server_key_version: "encryptionkey@0"
I20260811 05:13:27.529280 11195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:27.553330 11195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:27.556496 11195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:27.556720 11195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:27.564112 11195 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:34077
I20260811 05:13:27.564128 11300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:34077 every 8 connection(s)
I20260811 05:13:27.565138 11195 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:27.567977 11301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:27.573019 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 11195
I20260811 05:13:27.573149 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:13:27.573439 7438 external_mini_cluster.cc:1428] Setting key 50eede3bb7e1527c2ba004d42f46479a
I20260811 05:13:27.573698 11301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eac7edf5a2db4e1f9e8843bfa72d6fb6: Bootstrap starting.
I20260811 05:13:27.576269 11301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eac7edf5a2db4e1f9e8843bfa72d6fb6: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:27.577302 11301 log.cc:824] T 00000000000000000000000000000000 P eac7edf5a2db4e1f9e8843bfa72d6fb6: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:27.579409 11301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eac7edf5a2db4e1f9e8843bfa72d6fb6: No bootstrap required, opened a new log
Aug 11 05:13:27 dist-test-slave-04bb krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425207, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:27.583863 11301 raft_consensus.cc:348] T 00000000000000000000000000000000 P eac7edf5a2db4e1f9e8843bfa72d6fb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac7edf5a2db4e1f9e8843bfa72d6fb6" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 34077 } }
I20260811 05:13:27.584107 11301 raft_consensus.cc:374] T 00000000000000000000000000000000 P eac7edf5a2db4e1f9e8843bfa72d6fb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:27.584190 11301 raft_consensus.cc:729] T 00000000000000000000000000000000 P eac7edf5a2db4e1f9e8843bfa72d6fb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eac7edf5a2db4e1f9e8843bfa72d6fb6, State: Initialized, Role: FOLLOWER
I20260811 05:13:27.584726 11301 consensus_queue.cc:260] T 00000000000000000000000000000000 P eac7edf5a2db4e1f9e8843bfa72d6fb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac7edf5a2db4e1f9e8843bfa72d6fb6" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 34077 } }
I20260811 05:13:27.584901 11301 raft_consensus.cc:388] T 00000000000000000000000000000000 P eac7edf5a2db4e1f9e8843bfa72d6fb6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:27.584970 11301 raft_consensus.cc:482] T 00000000000000000000000000000000 P eac7edf5a2db4e1f9e8843bfa72d6fb6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:27.585073 11301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eac7edf5a2db4e1f9e8843bfa72d6fb6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:27.586151 11301 raft_consensus.cc:504] T 00000000000000000000000000000000 P eac7edf5a2db4e1f9e8843bfa72d6fb6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac7edf5a2db4e1f9e8843bfa72d6fb6" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 34077 } }
I20260811 05:13:27.586526 11301 leader_election.cc:302] T 00000000000000000000000000000000 P eac7edf5a2db4e1f9e8843bfa72d6fb6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eac7edf5a2db4e1f9e8843bfa72d6fb6; no voters:
I20260811 05:13:27.586864 11301 leader_election.cc:288] T 00000000000000000000000000000000 P eac7edf5a2db4e1f9e8843bfa72d6fb6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:27.586969 11311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eac7edf5a2db4e1f9e8843bfa72d6fb6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:27.587205 11311 raft_consensus.cc:686] T 00000000000000000000000000000000 P eac7edf5a2db4e1f9e8843bfa72d6fb6 [term 1 LEADER]: Becoming Leader. State: Replica: eac7edf5a2db4e1f9e8843bfa72d6fb6, State: Running, Role: LEADER
I20260811 05:13:27.587522 11311 consensus_queue.cc:237] T 00000000000000000000000000000000 P eac7edf5a2db4e1f9e8843bfa72d6fb6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac7edf5a2db4e1f9e8843bfa72d6fb6" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 34077 } }
I20260811 05:13:27.588083 11301 sys_catalog.cc:564] T 00000000000000000000000000000000 P eac7edf5a2db4e1f9e8843bfa72d6fb6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:27.588652 11305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:27.574779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50946 (local address 127.7.67.190:34077)
0811 05:13:27.575216 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:27.575233 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:27.575272 (+ 39us) server_negotiation.cc:407] Connection header received
0811 05:13:27.575898 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:27.575916 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:27.576233 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:27.576615 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:27.577515 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:27.578510 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:27.579640 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:27.580098 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:27.580216 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:27.583211 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:27.583217 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:27.583230 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:27.583263 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:27.585372 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:27.585800 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:27.585809 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:27.585819 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:27.585970 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:27.586238 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:27.586244 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:27.586248 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:27.586755 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:27.586954 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:27.587364 (+ 410us) server_negotiation.cc:299] Negotiation successful
0811 05:13:27.587670 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":288,"thread_start_us":149,"threads_started":1}
I20260811 05:13:27.588774 11311 sys_catalog.cc:455] T 00000000000000000000000000000000 P eac7edf5a2db4e1f9e8843bfa72d6fb6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eac7edf5a2db4e1f9e8843bfa72d6fb6. Latest consensus state: current_term: 1 leader_uuid: "eac7edf5a2db4e1f9e8843bfa72d6fb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac7edf5a2db4e1f9e8843bfa72d6fb6" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 34077 } } }
I20260811 05:13:27.588879 11311 sys_catalog.cc:458] T 00000000000000000000000000000000 P eac7edf5a2db4e1f9e8843bfa72d6fb6 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:27.589552 11312 sys_catalog.cc:455] T 00000000000000000000000000000000 P eac7edf5a2db4e1f9e8843bfa72d6fb6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eac7edf5a2db4e1f9e8843bfa72d6fb6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eac7edf5a2db4e1f9e8843bfa72d6fb6" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 34077 } } }
I20260811 05:13:27.589654 11312 sys_catalog.cc:458] T 00000000000000000000000000000000 P eac7edf5a2db4e1f9e8843bfa72d6fb6 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:27.589919 11319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:27.593093 11319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:27.598438 11319 catalog_manager.cc:1347] Generated new cluster ID: 151a7034e7b940e59be44793d4f49847
I20260811 05:13:27.598500 11319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:27.616609 11319 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:27.617790 11319 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:27.622514 11319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eac7edf5a2db4e1f9e8843bfa72d6fb6: Generated new TSK 0
I20260811 05:13:27.623251 11319 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
I20260811 05:13:27.655855 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:34077
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:46157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:27.771591 11336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:27.771831 11336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:27.771870 11336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:27.775800 11336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:27.775856 11336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:27.775942 11336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:13:27.781033 11336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:46157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:34077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11336
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:27.782056 11336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:27.783077 11336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:27.793676 11342 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:27.793808 11336 server_base.cc:1034] running on GCE node
W20260811 05:13:27.793641 11344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:27.793672 11341 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:27.794361 11336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:27.795020 11336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:27.796221 11336 hybrid_clock.cc:648] HybridClock initialized: now 1786425207796182 us; error 50 us; skew 500 ppm
Aug 11 05:13:27 dist-test-slave-04bb krb5kdc[11177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425207, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:27.799074 11336 init.cc:377] Logged in from keytab as kudu/127.7.67.129@KRBTEST.COM (short username kudu)
I20260811 05:13:27.799945 11336 webserver.cc:460] Webserver started at http://127.7.67.129:41881/ using document root <none> and password file <none>
I20260811 05:13:27.800710 11336 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:27.800797 11336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:27.801132 11336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:27.803311 11336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "d7146c1108524fc6848473f545aba1f4"
format_stamp: "Formatted at 2026-08-11 05:13:27 on dist-test-slave-04bb"
server_key: "66aa412cff468f179d40d7e508c1a17e"
server_key_iv: "c2e365b1fe5841b5edf562acc40c847f"
server_key_version: "encryptionkey@0"
I20260811 05:13:27.803876 11336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "d7146c1108524fc6848473f545aba1f4"
format_stamp: "Formatted at 2026-08-11 05:13:27 on dist-test-slave-04bb"
server_key: "66aa412cff468f179d40d7e508c1a17e"
server_key_iv: "c2e365b1fe5841b5edf562acc40c847f"
server_key_version: "encryptionkey@0"
I20260811 05:13:27.807662 11336 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 05:13:27.810477 11358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:27.811942 11336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 05:13:27.812072 11336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "d7146c1108524fc6848473f545aba1f4"
format_stamp: "Formatted at 2026-08-11 05:13:27 on dist-test-slave-04bb"
server_key: "66aa412cff468f179d40d7e508c1a17e"
server_key_iv: "c2e365b1fe5841b5edf562acc40c847f"
server_key_version: "encryptionkey@0"
I20260811 05:13:27.812191 11336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:27.828787 11336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:27.832458 11336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:27.832713 11336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:27.833411 11336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:27.834364 11336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:27.834429 11336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:27.834474 11336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:27.834499 11336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:27.844398 11336 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:33259
I20260811 05:13:27.844434 11530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:33259 every 8 connection(s)
I20260811 05:13:27.845368 11336 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:27.852731 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 11336
I20260811 05:13:27.852864 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:13:27.853127 7438 external_mini_cluster.cc:1428] Setting key 4c806b06d56ca53db76afdcf22eb8b54
Aug 11 05:13:27 dist-test-slave-04bb krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425207, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:27.860303 11305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:27.847393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:40003 (local address 127.7.67.190:34077)
0811 05:13:27.847550 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:27.847554 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:27.848665 (+ 1111us) server_negotiation.cc:407] Connection header received
0811 05:13:27.849627 (+ 962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:27.849631 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:27.849692 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:27.849808 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:27.850989 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:27.851554 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:27.852910 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:27.853210 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:27.853375 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:27.856114 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:27.856120 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:27.856126 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:27.856192 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:27.857825 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:27.858521 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:27.858524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:27.858525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:27.858604 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:27.859013 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:27.859016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:27.859017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:27.859232 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:27.859312 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:27.860046 (+ 734us) server_negotiation.cc:299] Negotiation successful
0811 05:13:27.860158 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 05:13:27.861558 11538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:27.847757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:34077 (local address 127.7.67.129:40003)
0811 05:13:27.848462 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:27.848506 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:27.849409 (+ 903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:27.849983 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:27.849990 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:27.850361 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:27.850837 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:27.850850 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:27.851696 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:27.851699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:27.852747 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:27.852761 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:27.853419 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:27.853423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:27.853619 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:27.854386 (+ 767us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:27.854412 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:27.855938 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:27.857950 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:27.857958 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:27.857976 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:27.858410 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:27.858704 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:27.858710 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:27.858715 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:27.858913 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:27.859345 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:27.859355 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:27.859721 (+ 366us) client_negotiation.cc:769] Sending connection context
0811 05:13:27.859953 (+ 232us) client_negotiation.cc:240] Negotiation successful
0811 05:13:27.860348 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":109,"threads_started":1}
I20260811 05:13:27.862987 11531 heartbeater.cc:347] Connected to a master server at 127.7.67.190:34077
I20260811 05:13:27.863283 11531 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:27.863835 11531 heartbeater.cc:511] Master 127.7.67.190:34077 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.67.130@KRBTEST.COM; defaulting to no policy
I20260811 05:13:27.865513 11245 ts_manager.cc:194] Registered new tserver with Master: d7146c1108524fc6848473f545aba1f4 (127.7.67.129:33259)
I20260811 05:13:27.866717 11245 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.129@KRBTEST.COM'} at 127.7.67.129:40003
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
I20260811 05:13:27.884506 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:34077
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:46157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:27.999409 11544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:27.999696 11544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:27.999768 11544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:28.003819 11544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:28.003916 11544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:28.004007 11544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:13:28.009289 11544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:46157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:34077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:28.010399 11544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:28.011453 11544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:28.021708 11557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:28.021793 11559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:28.021749 11556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:28.022234 11544 server_base.cc:1034] running on GCE node
I20260811 05:13:28.022591 11544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:28.023167 11544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:28.024329 11544 hybrid_clock.cc:648] HybridClock initialized: now 1786425208024309 us; error 30 us; skew 500 ppm
Aug 11 05:13:28 dist-test-slave-04bb krb5kdc[11177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425208, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:28.027240 11544 init.cc:377] Logged in from keytab as kudu/127.7.67.130@KRBTEST.COM (short username kudu)
I20260811 05:13:28.028121 11544 webserver.cc:460] Webserver started at http://127.7.67.130:42387/ using document root <none> and password file <none>
I20260811 05:13:28.028730 11544 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:28.028823 11544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:28.029013 11544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:28.031019 11544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "522dc499200b4e2e8116ef69addd2d24"
format_stamp: "Formatted at 2026-08-11 05:13:28 on dist-test-slave-04bb"
server_key: "82f71484a8207e4372f39cb040fddc09"
server_key_iv: "cdf64ae7fd992aed0b1115d8a2365a21"
server_key_version: "encryptionkey@0"
I20260811 05:13:28.031535 11544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "522dc499200b4e2e8116ef69addd2d24"
format_stamp: "Formatted at 2026-08-11 05:13:28 on dist-test-slave-04bb"
server_key: "82f71484a8207e4372f39cb040fddc09"
server_key_iv: "cdf64ae7fd992aed0b1115d8a2365a21"
server_key_version: "encryptionkey@0"
I20260811 05:13:28.035401 11544 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260811 05:13:28.037832 11568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:28.039049 11544 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:28.039175 11544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "522dc499200b4e2e8116ef69addd2d24"
format_stamp: "Formatted at 2026-08-11 05:13:28 on dist-test-slave-04bb"
server_key: "82f71484a8207e4372f39cb040fddc09"
server_key_iv: "cdf64ae7fd992aed0b1115d8a2365a21"
server_key_version: "encryptionkey@0"
I20260811 05:13:28.039283 11544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:28.054482 11544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:28.057919 11544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:28.058125 11544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:28.058732 11544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:28.059588 11544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:28.059636 11544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:28.059708 11544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:28.059746 11544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:28.068629 11544 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:46313
I20260811 05:13:28.068632 11755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:46313 every 8 connection(s)
I20260811 05:13:28.069543 11544 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:28.071844 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 11544
I20260811 05:13:28.071971 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:13:28.072388 7438 external_mini_cluster.cc:1428] Setting key a8dd3eae820a546958d9b69a6ad7f623
Aug 11 05:13:28 dist-test-slave-04bb krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425208, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.131@KRBTEST.COM; defaulting to no policy
I20260811 05:13:28.084199 11305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:28.071423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:60937 (local address 127.7.67.190:34077)
0811 05:13:28.071640 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:28.071645 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:28.072846 (+ 1201us) server_negotiation.cc:407] Connection header received
0811 05:13:28.073858 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:28.073861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:28.073914 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:28.073996 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:28.075173 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:28.075707 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:28.076927 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:28.077157 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:28.077285 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:28.080170 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:28.080176 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:28.080180 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:28.080213 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:28.081976 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:28.082501 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:28.082507 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:28.082510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:28.082612 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:28.082959 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:28.082963 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:28.082965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:28.083213 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:28.083347 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:28.083846 (+ 499us) server_negotiation.cc:299] Negotiation successful
0811 05:13:28.083981 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 05:13:28.085016 11761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:28.071812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:34077 (local address 127.7.67.130:60937)
0811 05:13:28.072634 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:28.072693 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:28.073608 (+ 915us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:28.074190 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:28.074197 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:28.074546 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:28.075024 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:28.075038 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:28.075842 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:28.075850 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:28.076797 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:28.076807 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:28.077314 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:28.077319 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:28.077551 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:28.078195 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:28.078220 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:28.079990 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:28.082094 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:28.082099 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:28.082111 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:28.082387 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:28.082721 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:28.082723 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:28.082724 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:28.082855 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:28.083350 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:28.083355 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:28.083586 (+ 231us) client_negotiation.cc:769] Sending connection context
0811 05:13:28.083747 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 05:13:28.083999 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":149,"threads_started":1}
I20260811 05:13:28.086452 11756 heartbeater.cc:347] Connected to a master server at 127.7.67.190:34077
I20260811 05:13:28.086762 11756 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.131@KRBTEST.COM" created.
I20260811 05:13:28.087399 11756 heartbeater.cc:511] Master 127.7.67.190:34077 requested a full tablet report, sending...
I20260811 05:13:28.088697 11245 ts_manager.cc:194] Registered new tserver with Master: 522dc499200b4e2e8116ef69addd2d24 (127.7.67.130:46313)
I20260811 05:13:28.089534 11245 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.130@KRBTEST.COM'} at 127.7.67.130:60937
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
I20260811 05:13:28.108939 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:34077
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:46157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:28.227968 11767 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:28.228240 11767 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:28.228286 11767 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:28.232414 11767 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:28.232481 11767 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:28.232658 11767 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:13:28.238060 11767 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:46157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:34077
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11767
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:28.239159 11767 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:28.240253 11767 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:28.250797 11780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:28.250929 11767 server_base.cc:1034] running on GCE node
W20260811 05:13:28.250799 11776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:28.250820 11777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:28.251619 11767 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:28.252300 11767 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:28.253521 11767 hybrid_clock.cc:648] HybridClock initialized: now 1786425208253517 us; error 62 us; skew 500 ppm
Aug 11 05:13:28 dist-test-slave-04bb krb5kdc[11177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425208, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:28.256702 11767 init.cc:377] Logged in from keytab as kudu/127.7.67.131@KRBTEST.COM (short username kudu)
I20260811 05:13:28.257566 11767 webserver.cc:460] Webserver started at http://127.7.67.131:36705/ using document root <none> and password file <none>
I20260811 05:13:28.258102 11767 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:28.258176 11767 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:28.258351 11767 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:28.260358 11767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "0118bdfab84340e5b2552f691a694398"
format_stamp: "Formatted at 2026-08-11 05:13:28 on dist-test-slave-04bb"
server_key: "cd9229aa9fca87babb78915cc9e1a6d6"
server_key_iv: "0ab337a115bdd5f34cc1db89e9852728"
server_key_version: "encryptionkey@0"
I20260811 05:13:28.261001 11767 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "0118bdfab84340e5b2552f691a694398"
format_stamp: "Formatted at 2026-08-11 05:13:28 on dist-test-slave-04bb"
server_key: "cd9229aa9fca87babb78915cc9e1a6d6"
server_key_iv: "0ab337a115bdd5f34cc1db89e9852728"
server_key_version: "encryptionkey@0"
I20260811 05:13:28.264959 11767 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 05:13:28.267462 11793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:28.268823 11767 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:13:28.268923 11767 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "0118bdfab84340e5b2552f691a694398"
format_stamp: "Formatted at 2026-08-11 05:13:28 on dist-test-slave-04bb"
server_key: "cd9229aa9fca87babb78915cc9e1a6d6"
server_key_iv: "0ab337a115bdd5f34cc1db89e9852728"
server_key_version: "encryptionkey@0"
I20260811 05:13:28.269009 11767 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:28.292963 11767 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:28.296306 11767 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:28.296463 11767 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:28.297111 11767 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:28.297905 11767 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:28.297951 11767 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:28.297991 11767 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:28.298007 11767 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:28.307166 11767 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:45189
I20260811 05:13:28.307276 11970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:45189 every 8 connection(s)
I20260811 05:13:28.308106 11767 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:13:28.316761 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 11767
I20260811 05:13:28.316916 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:13:28.317255 7438 external_mini_cluster.cc:1428] Setting key e7b80380b5e0ad909152bb76e3cb8cfc
Aug 11 05:13:28 dist-test-slave-04bb krb5kdc[11177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425208, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:28.324062 11305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:28.310097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:51001 (local address 127.7.67.190:34077)
0811 05:13:28.310240 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:28.310245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:28.311379 (+ 1134us) server_negotiation.cc:407] Connection header received
0811 05:13:28.312280 (+ 901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:28.312283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:28.312336 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:28.312424 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:28.313651 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:28.314252 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:28.315376 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:28.315620 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:28.315717 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:28.318613 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:28.318619 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:28.318623 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:28.318673 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:28.320711 (+ 2038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:28.321441 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:28.321445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:28.321447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:28.321546 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:28.322735 (+ 1189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:28.322739 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:28.322741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:28.323043 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:28.323194 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:28.323744 (+ 550us) server_negotiation.cc:299] Negotiation successful
0811 05:13:28.323877 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 05:13:28.324976 11974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:28.310573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:34077 (local address 127.7.67.131:51001)
0811 05:13:28.311242 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:28.311294 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:28.312047 (+ 753us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:28.312650 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:28.312657 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:28.313014 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:28.313491 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:28.313504 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:28.314375 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:28.314378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:28.315240 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:28.315251 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:28.315771 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:28.315774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:28.315913 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:28.316639 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:28.316655 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:28.318410 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:28.320863 (+ 2453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:28.320869 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:28.320884 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:28.321326 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:28.321675 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:28.321681 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:28.321684 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:28.322584 (+ 900us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:28.323194 (+ 610us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:28.323202 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:28.323466 (+ 264us) client_negotiation.cc:769] Sending connection context
0811 05:13:28.323681 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 05:13:28.324023 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":120,"threads_started":1}
I20260811 05:13:28.326139 11971 heartbeater.cc:347] Connected to a master server at 127.7.67.190:34077
I20260811 05:13:28.326393 11971 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:28.326884 11971 heartbeater.cc:511] Master 127.7.67.190:34077 requested a full tablet report, sending...
I20260811 05:13:28.327940 11245 ts_manager.cc:194] Registered new tserver with Master: 0118bdfab84340e5b2552f691a694398 (127.7.67.131:45189)
I20260811 05:13:28.328653 11245 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.131@KRBTEST.COM'} at 127.7.67.131:51001
I20260811 05:13:28.331545 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 05:13:28.346537 11305 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:28.344803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50956 (local address 127.7.67.190:34077)
0811 05:13:28.344976 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:28.344982 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:28.345197 (+ 215us) server_negotiation.cc:407] Connection header received
0811 05:13:28.345327 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:28.345333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:28.345384 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:28.345502 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:28.346361 (+ 859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50956: BlockingRecv error: recv got EOF from 127.0.0.1:50956 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:13:28.347765 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 11336
I20260811 05:13:28.375777 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 11544
I20260811 05:13:28.385854 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 11767
I20260811 05:13:28.398427 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 11195
2026-08-11T05:13:28Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2427 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:13:28 dist-test-slave-04bb krb5kdc[11994](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:13:28 dist-test-slave-04bb krb5kdc[11994](info): set up 2 sockets
Aug 11 05:13:28 dist-test-slave-04bb krb5kdc[11994](info): commencing operation
krb5kdc: starting...
W20260811 05:13:29.688215 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.260s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:13:29 dist-test-slave-04bb krb5kdc[11994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:13:29.820751 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:34477
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:44857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:34477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:29.937422 12015 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:29.937728 12015 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:29.937822 12015 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:29.941782 12015 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:29.941870 12015 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:29.941908 12015 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:29.941944 12015 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:29.941974 12015 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:29.947125 12015 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:44857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:34477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:34477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12015
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:29.948196 12015 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:29.949245 12015 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:29.957763 12021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:29.957713 12022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:29.957749 12025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:29.957939 12015 server_base.cc:1034] running on GCE node
I20260811 05:13:29.958559 12015 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:29.959651 12015 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:29.960845 12015 hybrid_clock.cc:648] HybridClock initialized: now 1786425209960825 us; error 31 us; skew 500 ppm
Aug 11 05:13:29 dist-test-slave-04bb krb5kdc[11994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425209, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:29.963598 12015 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:13:29.964461 12015 webserver.cc:460] Webserver started at http://127.7.67.190:46035/ using document root <none> and password file <none>
I20260811 05:13:29.965042 12015 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:29.965137 12015 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:29.965385 12015 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:29.967447 12015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "5e31b6a0984e4a848c21e6824f7aea95"
format_stamp: "Formatted at 2026-08-11 05:13:29 on dist-test-slave-04bb"
server_key: "4ea32dcc1e94690e4786a92de601144d"
server_key_iv: "65784a1173d5bc1b9d5efb09538e79b6"
server_key_version: "encryptionkey@0"
I20260811 05:13:29.968040 12015 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "5e31b6a0984e4a848c21e6824f7aea95"
format_stamp: "Formatted at 2026-08-11 05:13:29 on dist-test-slave-04bb"
server_key: "4ea32dcc1e94690e4786a92de601144d"
server_key_iv: "65784a1173d5bc1b9d5efb09538e79b6"
server_key_version: "encryptionkey@0"
I20260811 05:13:29.971786 12015 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 05:13:29.974319 12038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:29.975570 12015 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:29.975731 12015 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "5e31b6a0984e4a848c21e6824f7aea95"
format_stamp: "Formatted at 2026-08-11 05:13:29 on dist-test-slave-04bb"
server_key: "4ea32dcc1e94690e4786a92de601144d"
server_key_iv: "65784a1173d5bc1b9d5efb09538e79b6"
server_key_version: "encryptionkey@0"
I20260811 05:13:29.975860 12015 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:29.989936 12015 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:29.993209 12015 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:29.993430 12015 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:30.001024 12015 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:34477
I20260811 05:13:30.001442 12124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:34477 every 8 connection(s)
I20260811 05:13:30.002041 12015 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:30.004969 12125 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:30.008090 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 12015
I20260811 05:13:30.008239 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:13:30.008566 7438 external_mini_cluster.cc:1428] Setting key 648907e634be43246dac8307cc2b3e67
I20260811 05:13:30.011276 12125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e31b6a0984e4a848c21e6824f7aea95: Bootstrap starting.
I20260811 05:13:30.013962 12125 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5e31b6a0984e4a848c21e6824f7aea95: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:30.014717 12125 log.cc:824] T 00000000000000000000000000000000 P 5e31b6a0984e4a848c21e6824f7aea95: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:30.016840 12125 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5e31b6a0984e4a848c21e6824f7aea95: No bootstrap required, opened a new log
Aug 11 05:13:30 dist-test-slave-04bb krb5kdc[11994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425209, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:30.020143 12125 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5e31b6a0984e4a848c21e6824f7aea95 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e31b6a0984e4a848c21e6824f7aea95" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 34477 } }
I20260811 05:13:30.020406 12125 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5e31b6a0984e4a848c21e6824f7aea95 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:30.020503 12125 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5e31b6a0984e4a848c21e6824f7aea95 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e31b6a0984e4a848c21e6824f7aea95, State: Initialized, Role: FOLLOWER
I20260811 05:13:30.020960 12125 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5e31b6a0984e4a848c21e6824f7aea95 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e31b6a0984e4a848c21e6824f7aea95" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 34477 } }
I20260811 05:13:30.021112 12125 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5e31b6a0984e4a848c21e6824f7aea95 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:30.021200 12125 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5e31b6a0984e4a848c21e6824f7aea95 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:30.021328 12125 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5e31b6a0984e4a848c21e6824f7aea95 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:30.022195 12125 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5e31b6a0984e4a848c21e6824f7aea95 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e31b6a0984e4a848c21e6824f7aea95" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 34477 } }
I20260811 05:13:30.022547 12125 leader_election.cc:302] T 00000000000000000000000000000000 P 5e31b6a0984e4a848c21e6824f7aea95 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5e31b6a0984e4a848c21e6824f7aea95; no voters:
I20260811 05:13:30.022899 12125 leader_election.cc:288] T 00000000000000000000000000000000 P 5e31b6a0984e4a848c21e6824f7aea95 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:30.022970 12133 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5e31b6a0984e4a848c21e6824f7aea95 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:30.023191 12133 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5e31b6a0984e4a848c21e6824f7aea95 [term 1 LEADER]: Becoming Leader. State: Replica: 5e31b6a0984e4a848c21e6824f7aea95, State: Running, Role: LEADER
I20260811 05:13:30.023527 12133 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5e31b6a0984e4a848c21e6824f7aea95 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e31b6a0984e4a848c21e6824f7aea95" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 34477 } }
I20260811 05:13:30.024247 12125 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5e31b6a0984e4a848c21e6824f7aea95 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:30.024340 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:30.009956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59626 (local address 127.7.67.190:34477)
0811 05:13:30.010386 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:30.010402 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:30.010437 (+ 35us) server_negotiation.cc:407] Connection header received
0811 05:13:30.011256 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:30.011273 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:30.011668 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:30.011948 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:30.012949 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:30.013973 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:30.014851 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:30.015496 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:30.015688 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:30.018313 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:30.018319 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:30.018332 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:30.018359 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:30.020930 (+ 2571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:30.021487 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:30.021494 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:30.021500 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:30.021624 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:30.021931 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:30.021934 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:30.021936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:30.022364 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:30.022570 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:30.022890 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 05:13:30.023466 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":143,"threads_started":1}
I20260811 05:13:30.025606 12138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e31b6a0984e4a848c21e6824f7aea95 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5e31b6a0984e4a848c21e6824f7aea95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e31b6a0984e4a848c21e6824f7aea95" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 34477 } } }
I20260811 05:13:30.026679 12138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e31b6a0984e4a848c21e6824f7aea95 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:30.025597 12137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5e31b6a0984e4a848c21e6824f7aea95 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5e31b6a0984e4a848c21e6824f7aea95. Latest consensus state: current_term: 1 leader_uuid: "5e31b6a0984e4a848c21e6824f7aea95" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5e31b6a0984e4a848c21e6824f7aea95" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 34477 } } }
I20260811 05:13:30.027488 12137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5e31b6a0984e4a848c21e6824f7aea95 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:30.027949 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260811 05:13:30.030526 12155 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5e31b6a0984e4a848c21e6824f7aea95: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:13:30.030671 12155 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:13:30.030782 12158 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:30.031466 12158 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:30.037998 12158 catalog_manager.cc:1347] Generated new cluster ID: 2d1a94ddf3284e05ae69f53469c64534
I20260811 05:13:30.038062 12158 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:30.047799 12130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:30.042715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59630 (local address 127.7.67.190:34477)
0811 05:13:30.042834 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:30.042839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:30.043093 (+ 254us) server_negotiation.cc:407] Connection header received
0811 05:13:30.043259 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:30.043262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:30.043313 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:30.043405 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:30.043917 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:30.044508 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:30.045431 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:30.045683 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:30.045818 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:30.046718 (+ 900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:30.046722 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:30.046725 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:30.047274 (+ 549us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:30.047310 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:30.047330 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:30.047422 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:30.047586 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 05:13:30.047648 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":46}
I20260811 05:13:30.047920 12158 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:30.049153 12158 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:30.060868 12158 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5e31b6a0984e4a848c21e6824f7aea95: Generated new TSK 0
I20260811 05:13:30.061568 12158 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:13:30.068225 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 12015
2026-08-11T05:13:30Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1673 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:13:30 dist-test-slave-04bb krb5kdc[12177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:13:30 dist-test-slave-04bb krb5kdc[12177](info): set up 2 sockets
Aug 11 05:13:30 dist-test-slave-04bb krb5kdc[12177](info): commencing operation
krb5kdc: starting...
W20260811 05:13:31.348881 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:13:31 dist-test-slave-04bb krb5kdc[12177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:13:31.480262 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:42979
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:45307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:42979
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:31.595355 12198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:31.595674 12198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:31.595757 12198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:31.599805 12198 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:31.599900 12198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:31.599939 12198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:31.599977 12198 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:31.600011 12198 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:31.605360 12198 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:45307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:42979
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:42979
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:31.606564 12198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:31.607617 12198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:31.616245 12207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:31.616212 12206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:31.616454 12198 server_base.cc:1034] running on GCE node
W20260811 05:13:31.616277 12212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:31.617197 12198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:31.618304 12198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:31.619514 12198 hybrid_clock.cc:648] HybridClock initialized: now 1786425211619484 us; error 47 us; skew 500 ppm
Aug 11 05:13:31 dist-test-slave-04bb krb5kdc[12177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425211, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:31.627405 12198 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:13:31.628358 12198 webserver.cc:460] Webserver started at http://127.7.67.190:46367/ using document root <none> and password file <none>
I20260811 05:13:31.629004 12198 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:31.629102 12198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:31.629354 12198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:31.631445 12198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "2f52cd5a41104585acb18d8b75a8de6c"
format_stamp: "Formatted at 2026-08-11 05:13:31 on dist-test-slave-04bb"
server_key: "07bbd5ece46f4757ff4c8005bdeae327"
server_key_iv: "3aeb8c887a5de704d528583972828521"
server_key_version: "encryptionkey@0"
I20260811 05:13:31.632040 12198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "2f52cd5a41104585acb18d8b75a8de6c"
format_stamp: "Formatted at 2026-08-11 05:13:31 on dist-test-slave-04bb"
server_key: "07bbd5ece46f4757ff4c8005bdeae327"
server_key_iv: "3aeb8c887a5de704d528583972828521"
server_key_version: "encryptionkey@0"
I20260811 05:13:31.635931 12198 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 05:13:31.638625 12225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:31.639902 12198 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:13:31.640057 12198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "2f52cd5a41104585acb18d8b75a8de6c"
format_stamp: "Formatted at 2026-08-11 05:13:31 on dist-test-slave-04bb"
server_key: "07bbd5ece46f4757ff4c8005bdeae327"
server_key_iv: "3aeb8c887a5de704d528583972828521"
server_key_version: "encryptionkey@0"
I20260811 05:13:31.640177 12198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:31.649382 12198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:31.652629 12198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:31.652868 12198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:31.660780 12198 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:42979
I20260811 05:13:31.661082 12308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:42979 every 8 connection(s)
I20260811 05:13:31.661873 12198 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:31.664845 12311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:31.667148 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 12198
I20260811 05:13:31.667284 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:13:31.667570 7438 external_mini_cluster.cc:1428] Setting key 2d91ffc6ce456d7dd566aa2f97c0c90d
I20260811 05:13:31.671281 12311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f52cd5a41104585acb18d8b75a8de6c: Bootstrap starting.
I20260811 05:13:31.673593 12311 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f52cd5a41104585acb18d8b75a8de6c: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:31.674329 12311 log.cc:824] T 00000000000000000000000000000000 P 2f52cd5a41104585acb18d8b75a8de6c: Log is configured to *not* fsync() on all Append() calls
Aug 11 05:13:31 dist-test-slave-04bb krb5kdc[12177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:31.676492 12311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f52cd5a41104585acb18d8b75a8de6c: No bootstrap required, opened a new log
I20260811 05:13:31.679531 12311 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f52cd5a41104585acb18d8b75a8de6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f52cd5a41104585acb18d8b75a8de6c" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42979 } }
I20260811 05:13:31.679742 12311 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f52cd5a41104585acb18d8b75a8de6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:31.679822 12311 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f52cd5a41104585acb18d8b75a8de6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f52cd5a41104585acb18d8b75a8de6c, State: Initialized, Role: FOLLOWER
I20260811 05:13:31.680327 12311 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f52cd5a41104585acb18d8b75a8de6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f52cd5a41104585acb18d8b75a8de6c" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42979 } }
I20260811 05:13:31.680457 12311 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f52cd5a41104585acb18d8b75a8de6c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:31.680564 12311 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f52cd5a41104585acb18d8b75a8de6c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:31.680692 12311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f52cd5a41104585acb18d8b75a8de6c [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:31.681893 12311 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f52cd5a41104585acb18d8b75a8de6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f52cd5a41104585acb18d8b75a8de6c" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42979 } }
I20260811 05:13:31.682260 12311 leader_election.cc:302] T 00000000000000000000000000000000 P 2f52cd5a41104585acb18d8b75a8de6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f52cd5a41104585acb18d8b75a8de6c; no voters:
I20260811 05:13:31.682590 12311 leader_election.cc:288] T 00000000000000000000000000000000 P 2f52cd5a41104585acb18d8b75a8de6c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:31.682719 12322 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f52cd5a41104585acb18d8b75a8de6c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:31.682979 12322 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f52cd5a41104585acb18d8b75a8de6c [term 1 LEADER]: Becoming Leader. State: Replica: 2f52cd5a41104585acb18d8b75a8de6c, State: Running, Role: LEADER
I20260811 05:13:31.683394 12322 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f52cd5a41104585acb18d8b75a8de6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f52cd5a41104585acb18d8b75a8de6c" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42979 } }
I20260811 05:13:31.683853 12311 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f52cd5a41104585acb18d8b75a8de6c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:31.684369 12319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:31.669169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42356 (local address 127.7.67.190:42979)
0811 05:13:31.669630 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:31.669644 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:31.669680 (+ 36us) server_negotiation.cc:407] Connection header received
0811 05:13:31.670363 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:31.670382 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:31.670711 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:31.671067 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:31.671889 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:31.672919 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:31.673828 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:31.674309 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:31.674486 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:31.677264 (+ 2778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:31.677269 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:31.677282 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:31.677308 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:31.680884 (+ 3576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:31.681387 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:31.681392 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:31.681396 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:31.681520 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:31.681865 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:31.681867 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:31.681868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:31.682301 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:31.682514 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:31.682911 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 05:13:31.683408 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":139,"threads_started":1}
I20260811 05:13:31.684649 12322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f52cd5a41104585acb18d8b75a8de6c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f52cd5a41104585acb18d8b75a8de6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f52cd5a41104585acb18d8b75a8de6c" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42979 } } }
I20260811 05:13:31.684813 12322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f52cd5a41104585acb18d8b75a8de6c [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:31.685074 12322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f52cd5a41104585acb18d8b75a8de6c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f52cd5a41104585acb18d8b75a8de6c. Latest consensus state: current_term: 1 leader_uuid: "2f52cd5a41104585acb18d8b75a8de6c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f52cd5a41104585acb18d8b75a8de6c" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42979 } } }
I20260811 05:13:31.685227 12322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f52cd5a41104585acb18d8b75a8de6c [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:31.685528 12337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:31.688748 12337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:31.694962 12337 catalog_manager.cc:1347] Generated new cluster ID: c7ca082306494e0796b47770fff0183a
I20260811 05:13:31.695031 12337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:31.707974 12337 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:31.708992 12337 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:31.713768 12337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f52cd5a41104585acb18d8b75a8de6c: Generated new TSK 0
I20260811 05:13:31.714414 12337 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
I20260811 05:13:31.748350 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:42979
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:45307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:31.865720 12363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:31.865985 12363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:31.866029 12363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:31.870069 12363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:31.870132 12363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:31.870190 12363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:13:31.875698 12363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:45307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:42979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:31.876909 12363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:31.878016 12363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:31.888603 12374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:31.888651 12371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:31.888620 12372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:31.888913 12363 server_base.cc:1034] running on GCE node
I20260811 05:13:31.889355 12363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:31.889998 12363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:31.891165 12363 hybrid_clock.cc:648] HybridClock initialized: now 1786425211891146 us; error 34 us; skew 500 ppm
Aug 11 05:13:31 dist-test-slave-04bb krb5kdc[12177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425211, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:31.894449 12363 init.cc:377] Logged in from keytab as kudu/127.7.67.129@KRBTEST.COM (short username kudu)
I20260811 05:13:31.895428 12363 webserver.cc:460] Webserver started at http://127.7.67.129:45703/ using document root <none> and password file <none>
I20260811 05:13:31.896046 12363 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:31.896108 12363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:31.896349 12363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:31.898535 12363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "ccfdb410948a463cb2dd6961c2274614"
format_stamp: "Formatted at 2026-08-11 05:13:31 on dist-test-slave-04bb"
server_key: "29c7d34356a12bef60c6ff079f48b125"
server_key_iv: "770aea83711a8ab393426eea75936afc"
server_key_version: "encryptionkey@0"
I20260811 05:13:31.899132 12363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "ccfdb410948a463cb2dd6961c2274614"
format_stamp: "Formatted at 2026-08-11 05:13:31 on dist-test-slave-04bb"
server_key: "29c7d34356a12bef60c6ff079f48b125"
server_key_iv: "770aea83711a8ab393426eea75936afc"
server_key_version: "encryptionkey@0"
I20260811 05:13:31.903193 12363 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 05:13:31.906044 12382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:31.907418 12363 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 05:13:31.907646 12363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "ccfdb410948a463cb2dd6961c2274614"
format_stamp: "Formatted at 2026-08-11 05:13:31 on dist-test-slave-04bb"
server_key: "29c7d34356a12bef60c6ff079f48b125"
server_key_iv: "770aea83711a8ab393426eea75936afc"
server_key_version: "encryptionkey@0"
I20260811 05:13:31.907799 12363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:31.944931 12363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:31.948143 12363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:31.948362 12363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:31.949034 12363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:31.949909 12363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:31.949985 12363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:31.950062 12363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:31.950107 12363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:31.959277 12363 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:42783
I20260811 05:13:31.959332 12565 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:42783 every 8 connection(s)
I20260811 05:13:31.960193 12363 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:31.965914 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 12363
I20260811 05:13:31.966043 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:13:31.966398 7438 external_mini_cluster.cc:1428] Setting key 03edf9697c8b01c54aecd52db5629b0f
Aug 11 05:13:31 dist-test-slave-04bb krb5kdc[12177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425211, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:31.974612 12319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:31.962139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:59081 (local address 127.7.67.190:42979)
0811 05:13:31.962322 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:31.962327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:31.963155 (+ 828us) server_negotiation.cc:407] Connection header received
0811 05:13:31.964465 (+ 1310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:31.964468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:31.964515 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:31.964651 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:31.965828 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:31.966620 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:31.967992 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:31.968319 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:31.968432 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:31.970926 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:31.970935 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:31.970937 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:31.970960 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:31.972422 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:31.972978 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:31.972983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:31.972985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:31.973066 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:31.973474 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:31.973478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:31.973479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:31.973669 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:31.973782 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:31.974283 (+ 501us) server_negotiation.cc:299] Negotiation successful
0811 05:13:31.974440 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 05:13:31.975399 12569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:31.962531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42979 (local address 127.7.67.129:59081)
0811 05:13:31.962997 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:31.963041 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:31.964262 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:31.964826 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:31.964833 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:31.965174 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:31.965666 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:31.965680 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:31.966762 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:31.966766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:31.967892 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:31.967902 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:31.968473 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:31.968476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:31.968630 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:31.969259 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:31.969275 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:31.970759 (+ 1484us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:31.972569 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:31.972575 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:31.972589 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:31.972867 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:31.973165 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:31.973168 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:31.973170 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:31.973370 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:31.973782 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:31.973790 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:31.974037 (+ 247us) client_negotiation.cc:769] Sending connection context
0811 05:13:31.974231 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 05:13:31.974552 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":114,"threads_started":1}
I20260811 05:13:31.976831 12566 heartbeater.cc:347] Connected to a master server at 127.7.67.190:42979
WARNING: no policy specified for kudu/127.7.67.130@KRBTEST.COM; defaulting to no policy
I20260811 05:13:31.977115 12566 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:31.977667 12566 heartbeater.cc:511] Master 127.7.67.190:42979 requested a full tablet report, sending...
I20260811 05:13:31.979348 12255 ts_manager.cc:194] Registered new tserver with Master: ccfdb410948a463cb2dd6961c2274614 (127.7.67.129:42783)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.130@KRBTEST.COM" created.
I20260811 05:13:31.980829 12255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.129@KRBTEST.COM'} at 127.7.67.129:59081
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
I20260811 05:13:31.995903 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:42979
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:45307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:32.113643 12576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:32.113891 12576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:32.113966 12576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:32.117983 12576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:32.118036 12576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:32.118129 12576 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:13:32.123284 12576 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:45307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:42979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:32.124332 12576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:32.125407 12576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:32.135160 12584 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:32.135147 12583 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:32.135150 12587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:32.135471 12576 server_base.cc:1034] running on GCE node
I20260811 05:13:32.135962 12576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:32.136629 12576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:32.137826 12576 hybrid_clock.cc:648] HybridClock initialized: now 1786425212137812 us; error 25 us; skew 500 ppm
Aug 11 05:13:32 dist-test-slave-04bb krb5kdc[12177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425212, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:32.140478 12576 init.cc:377] Logged in from keytab as kudu/127.7.67.130@KRBTEST.COM (short username kudu)
I20260811 05:13:32.141378 12576 webserver.cc:460] Webserver started at http://127.7.67.130:45439/ using document root <none> and password file <none>
I20260811 05:13:32.141944 12576 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:32.142005 12576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:32.142273 12576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:32.144599 12576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "5637637bff8f4377a108656c41d06894"
format_stamp: "Formatted at 2026-08-11 05:13:32 on dist-test-slave-04bb"
server_key: "32cef4ef6531f2854111ca4f370b374f"
server_key_iv: "67da27b870a98dcc00458bd9dd5239a2"
server_key_version: "encryptionkey@0"
I20260811 05:13:32.145179 12576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "5637637bff8f4377a108656c41d06894"
format_stamp: "Formatted at 2026-08-11 05:13:32 on dist-test-slave-04bb"
server_key: "32cef4ef6531f2854111ca4f370b374f"
server_key_iv: "67da27b870a98dcc00458bd9dd5239a2"
server_key_version: "encryptionkey@0"
I20260811 05:13:32.149015 12576 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 05:13:32.151455 12598 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:32.152944 12576 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:13:32.153080 12576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "5637637bff8f4377a108656c41d06894"
format_stamp: "Formatted at 2026-08-11 05:13:32 on dist-test-slave-04bb"
server_key: "32cef4ef6531f2854111ca4f370b374f"
server_key_iv: "67da27b870a98dcc00458bd9dd5239a2"
server_key_version: "encryptionkey@0"
I20260811 05:13:32.153182 12576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:32.165586 12576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:32.168395 12576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:32.168613 12576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:32.169189 12576 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:32.170001 12576 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:32.170048 12576 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:32.170117 12576 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:32.170153 12576 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:32.179735 12576 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:39961
I20260811 05:13:32.179759 12776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:39961 every 8 connection(s)
I20260811 05:13:32.180688 12576 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:32.183070 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 12576
I20260811 05:13:32.183218 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:13:32.183531 7438 external_mini_cluster.cc:1428] Setting key 18e4dec54f1bd8af6b3be0651d211d65
Aug 11 05:13:32 dist-test-slave-04bb krb5kdc[12177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425212, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.131@KRBTEST.COM" created.
I20260811 05:13:32.197463 12319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:32.182554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:60013 (local address 127.7.67.190:42979)
0811 05:13:32.182697 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:32.182701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:32.184343 (+ 1642us) server_negotiation.cc:407] Connection header received
0811 05:13:32.185962 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:32.185968 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:32.186036 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:32.186122 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:32.187609 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:32.188485 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:32.189679 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:32.190117 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:32.190251 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:32.193254 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:32.193257 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:32.193259 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:32.193285 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:32.194906 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:32.195652 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:32.195656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:32.195657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:32.195735 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:32.196094 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:32.196098 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:32.196100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:32.196297 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:32.196415 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:32.197070 (+ 655us) server_negotiation.cc:299] Negotiation successful
0811 05:13:32.197297 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 05:13:32.198340 12781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:32.182907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42979 (local address 127.7.67.130:60013)
0811 05:13:32.183933 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:32.184041 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:32.185684 (+ 1643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:32.186325 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:32.186332 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:32.186932 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:32.187450 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:32.187463 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:32.188652 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:32.188655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:32.189544 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:32.189554 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:32.190326 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:32.190331 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:32.190507 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:32.191119 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:32.191136 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:32.193062 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:32.195047 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:32.195053 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:32.195067 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:32.195534 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:32.195842 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:32.195845 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:32.195847 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:32.195990 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:32.196414 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:32.196422 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:32.196800 (+ 378us) client_negotiation.cc:769] Sending connection context
0811 05:13:32.197015 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 05:13:32.197339 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"thread_start_us":158,"threads_started":1}
I20260811 05:13:32.199653 12779 heartbeater.cc:347] Connected to a master server at 127.7.67.190:42979
I20260811 05:13:32.199921 12779 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:32.200444 12779 heartbeater.cc:511] Master 127.7.67.190:42979 requested a full tablet report, sending...
I20260811 05:13:32.201849 12255 ts_manager.cc:194] Registered new tserver with Master: 5637637bff8f4377a108656c41d06894 (127.7.67.130:39961)
I20260811 05:13:32.202783 12255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.130@KRBTEST.COM'} at 127.7.67.130:60013
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
I20260811 05:13:32.220067 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:42979
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:45307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:32.333401 12785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:32.333621 12785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:32.333659 12785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:32.338790 12785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:32.338845 12785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:32.338900 12785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:13:32.344009 12785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:45307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:42979
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:32.345098 12785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:32.346179 12785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:32.355872 12795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:32.355988 12798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:32.356003 12794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:32.356181 12785 server_base.cc:1034] running on GCE node
I20260811 05:13:32.356715 12785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:32.357442 12785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:32.358641 12785 hybrid_clock.cc:648] HybridClock initialized: now 1786425212358610 us; error 49 us; skew 500 ppm
Aug 11 05:13:32 dist-test-slave-04bb krb5kdc[12177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425212, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:32.361451 12785 init.cc:377] Logged in from keytab as kudu/127.7.67.131@KRBTEST.COM (short username kudu)
I20260811 05:13:32.362257 12785 webserver.cc:460] Webserver started at http://127.7.67.131:41763/ using document root <none> and password file <none>
I20260811 05:13:32.362787 12785 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:32.362846 12785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:32.363014 12785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:32.365039 12785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "37e307fdf14042848a69893888720c2f"
format_stamp: "Formatted at 2026-08-11 05:13:32 on dist-test-slave-04bb"
server_key: "ba8be5c799ec12e13b060ac891f948fd"
server_key_iv: "8d4c3c03fe67943e5af16ff008e1800b"
server_key_version: "encryptionkey@0"
I20260811 05:13:32.365535 12785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "37e307fdf14042848a69893888720c2f"
format_stamp: "Formatted at 2026-08-11 05:13:32 on dist-test-slave-04bb"
server_key: "ba8be5c799ec12e13b060ac891f948fd"
server_key_iv: "8d4c3c03fe67943e5af16ff008e1800b"
server_key_version: "encryptionkey@0"
I20260811 05:13:32.369133 12785 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:13:32.371479 12812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:32.372812 12785 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:13:32.372906 12785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "37e307fdf14042848a69893888720c2f"
format_stamp: "Formatted at 2026-08-11 05:13:32 on dist-test-slave-04bb"
server_key: "ba8be5c799ec12e13b060ac891f948fd"
server_key_iv: "8d4c3c03fe67943e5af16ff008e1800b"
server_key_version: "encryptionkey@0"
I20260811 05:13:32.372989 12785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:32.388368 12785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:32.391645 12785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:32.391822 12785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:32.392373 12785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:32.393289 12785 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:32.393342 12785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:32.393381 12785 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:32.393397 12785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:32.402491 12785 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:46321
I20260811 05:13:32.402534 13011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:46321 every 8 connection(s)
I20260811 05:13:32.403497 12785 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:13:32.407131 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 12785
I20260811 05:13:32.407320 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:13:32.407671 7438 external_mini_cluster.cc:1428] Setting key 90a1cfedb3c638cb112c20e2bbd362d7
Aug 11 05:13:32 dist-test-slave-04bb krb5kdc[12177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425212, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:32.418015 12319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:32.405605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:48045 (local address 127.7.67.190:42979)
0811 05:13:32.405756 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:32.405761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:32.406624 (+ 863us) server_negotiation.cc:407] Connection header received
0811 05:13:32.408373 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:32.408377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:32.408434 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:32.408567 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:32.409859 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:32.410483 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:32.411648 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:32.411878 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:32.411992 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:32.414587 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:32.414590 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:32.414592 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:32.414616 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:32.416089 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:32.416615 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:32.416618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:32.416619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:32.416698 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:32.417022 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:32.417024 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:32.417025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:32.417239 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:32.417316 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:32.417795 (+ 479us) server_negotiation.cc:299] Negotiation successful
0811 05:13:32.417896 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 05:13:32.418839 13015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:32.406001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42979 (local address 127.7.67.131:48045)
0811 05:13:32.406468 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:32.406511 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:32.408152 (+ 1641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:32.408793 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:32.408802 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:32.409219 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:32.409708 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:32.409722 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:32.410621 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:32.410629 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:32.411500 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:32.411509 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:32.412031 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:32.412035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:32.412210 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:32.412892 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:32.412908 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:32.414432 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:32.416203 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:32.416208 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:32.416219 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:32.416469 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:32.416777 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:32.416779 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:32.416780 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:32.416928 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:32.417335 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:32.417340 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:32.417568 (+ 228us) client_negotiation.cc:769] Sending connection context
0811 05:13:32.417719 (+ 151us) client_negotiation.cc:240] Negotiation successful
0811 05:13:32.417982 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":115,"threads_started":1}
I20260811 05:13:32.419987 13013 heartbeater.cc:347] Connected to a master server at 127.7.67.190:42979
I20260811 05:13:32.420238 13013 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:32.420889 13013 heartbeater.cc:511] Master 127.7.67.190:42979 requested a full tablet report, sending...
I20260811 05:13:32.421979 12255 ts_manager.cc:194] Registered new tserver with Master: 37e307fdf14042848a69893888720c2f (127.7.67.131:46321)
I20260811 05:13:32.422529 12255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.131@KRBTEST.COM'} at 127.7.67.131:48045
I20260811 05:13:32.433641 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 05:13:32 dist-test-slave-04bb krb5kdc[12177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425212, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 05:13:32 dist-test-slave-04bb krb5kdc[12177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425212, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:32.461692 12319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:32.452270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42364 (local address 127.7.67.190:42979)
0811 05:13:32.452577 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:32.452582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:32.452735 (+ 153us) server_negotiation.cc:407] Connection header received
0811 05:13:32.452929 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:32.452932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:32.452976 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:32.453066 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:32.453833 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:32.454362 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:32.455227 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:32.455447 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:32.455563 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:32.458337 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:32.458340 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:32.458342 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:32.458365 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:32.459898 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:32.460355 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:32.460364 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:32.460365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:32.460440 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:32.460782 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:32.460785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:32.460787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:32.461006 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:32.461160 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:32.461452 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 05:13:32.461559 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178}
W20260811 05:13:32.462168 12255 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42364
I20260811 05:13:32.463362 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 12363
I20260811 05:13:32.479525 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 12576
I20260811 05:13:32.498580 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 12785
I20260811 05:13:32.513228 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 12198
2026-08-11T05:13:32Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2438 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:13:32 dist-test-slave-04bb krb5kdc[13027](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:13:32 dist-test-slave-04bb krb5kdc[13027](info): set up 2 sockets
Aug 11 05:13:32 dist-test-slave-04bb krb5kdc[13027](info): commencing operation
krb5kdc: starting...
W20260811 05:13:33.780627 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:13:33 dist-test-slave-04bb krb5kdc[13027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:13:33.912114 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:35943
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:37555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:35943
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:34.028855 13048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:34.029201 13048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:34.029307 13048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:34.033406 13048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:34.033491 13048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:34.033535 13048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:34.033599 13048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:34.033640 13048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:34.039579 13048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:37555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:35943
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:35943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:34.040759 13048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:34.041841 13048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:34.050812 13062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:34.050823 13056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:34.050812 13057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:34.051169 13048 server_base.cc:1034] running on GCE node
I20260811 05:13:34.051734 13048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:34.052893 13048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:34.054100 13048 hybrid_clock.cc:648] HybridClock initialized: now 1786425214054086 us; error 36 us; skew 500 ppm
Aug 11 05:13:34 dist-test-slave-04bb krb5kdc[13027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425214, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:34.056953 13048 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:13:34.057899 13048 webserver.cc:460] Webserver started at http://127.7.67.190:40327/ using document root <none> and password file <none>
I20260811 05:13:34.058547 13048 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:34.058643 13048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:34.058888 13048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:34.060977 13048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "e7d64c5db88940548bbc7e7ea3e7c8b0"
format_stamp: "Formatted at 2026-08-11 05:13:34 on dist-test-slave-04bb"
server_key: "d21cd2c52e52f6909cedaa200fbb56e0"
server_key_iv: "b1db9560980bc8419670eab30e8c8df7"
server_key_version: "encryptionkey@0"
I20260811 05:13:34.061558 13048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "e7d64c5db88940548bbc7e7ea3e7c8b0"
format_stamp: "Formatted at 2026-08-11 05:13:34 on dist-test-slave-04bb"
server_key: "d21cd2c52e52f6909cedaa200fbb56e0"
server_key_iv: "b1db9560980bc8419670eab30e8c8df7"
server_key_version: "encryptionkey@0"
I20260811 05:13:34.065443 13048 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 05:13:34.067915 13072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:34.069342 13048 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 05:13:34.069497 13048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "e7d64c5db88940548bbc7e7ea3e7c8b0"
format_stamp: "Formatted at 2026-08-11 05:13:34 on dist-test-slave-04bb"
server_key: "d21cd2c52e52f6909cedaa200fbb56e0"
server_key_iv: "b1db9560980bc8419670eab30e8c8df7"
server_key_version: "encryptionkey@0"
I20260811 05:13:34.069613 13048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:34.117295 13048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:34.120460 13048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:34.120709 13048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:34.127936 13048 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:35943
I20260811 05:13:34.127969 13157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:35943 every 8 connection(s)
I20260811 05:13:34.128925 13048 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:34.129506 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 13048
I20260811 05:13:34.129660 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:13:34.129966 7438 external_mini_cluster.cc:1428] Setting key f836f8ef0478dcbab6c7800a25917cca
I20260811 05:13:34.131968 13159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:34.138794 13159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d64c5db88940548bbc7e7ea3e7c8b0: Bootstrap starting.
Aug 11 05:13:34 dist-test-slave-04bb krb5kdc[13027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425213, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:34.141218 13159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e7d64c5db88940548bbc7e7ea3e7c8b0: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:34.141928 13159 log.cc:824] T 00000000000000000000000000000000 P e7d64c5db88940548bbc7e7ea3e7c8b0: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:34.144344 13159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e7d64c5db88940548bbc7e7ea3e7c8b0: No bootstrap required, opened a new log
I20260811 05:13:34.145116 13163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:34.131471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53558 (local address 127.7.67.190:35943)
0811 05:13:34.131900 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:34.131919 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:34.131964 (+ 45us) server_negotiation.cc:407] Connection header received
0811 05:13:34.132720 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:34.132738 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:34.133059 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:34.133350 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:34.134277 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.135258 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.136446 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.136995 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.137116 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:34.140021 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:34.140027 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:34.140040 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:34.140067 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:34.141912 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:34.142394 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:34.142398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:34.142403 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:34.142510 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:34.142885 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:34.142892 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:34.142896 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:34.143381 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:34.143575 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:34.143844 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 05:13:34.144141 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":157,"threads_started":1}
I20260811 05:13:34.148325 13159 raft_consensus.cc:348] T 00000000000000000000000000000000 P e7d64c5db88940548bbc7e7ea3e7c8b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d64c5db88940548bbc7e7ea3e7c8b0" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 35943 } }
I20260811 05:13:34.148581 13159 raft_consensus.cc:374] T 00000000000000000000000000000000 P e7d64c5db88940548bbc7e7ea3e7c8b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:34.148649 13159 raft_consensus.cc:729] T 00000000000000000000000000000000 P e7d64c5db88940548bbc7e7ea3e7c8b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7d64c5db88940548bbc7e7ea3e7c8b0, State: Initialized, Role: FOLLOWER
I20260811 05:13:34.149065 13159 consensus_queue.cc:260] T 00000000000000000000000000000000 P e7d64c5db88940548bbc7e7ea3e7c8b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d64c5db88940548bbc7e7ea3e7c8b0" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 35943 } }
I20260811 05:13:34.149190 13159 raft_consensus.cc:388] T 00000000000000000000000000000000 P e7d64c5db88940548bbc7e7ea3e7c8b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:34.149247 13159 raft_consensus.cc:482] T 00000000000000000000000000000000 P e7d64c5db88940548bbc7e7ea3e7c8b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:34.149320 13159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e7d64c5db88940548bbc7e7ea3e7c8b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:34.150091 13159 raft_consensus.cc:504] T 00000000000000000000000000000000 P e7d64c5db88940548bbc7e7ea3e7c8b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d64c5db88940548bbc7e7ea3e7c8b0" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 35943 } }
I20260811 05:13:34.150381 13159 leader_election.cc:302] T 00000000000000000000000000000000 P e7d64c5db88940548bbc7e7ea3e7c8b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e7d64c5db88940548bbc7e7ea3e7c8b0; no voters:
I20260811 05:13:34.150626 13159 leader_election.cc:288] T 00000000000000000000000000000000 P e7d64c5db88940548bbc7e7ea3e7c8b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:34.150879 13166 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e7d64c5db88940548bbc7e7ea3e7c8b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:34.151109 13166 raft_consensus.cc:686] T 00000000000000000000000000000000 P e7d64c5db88940548bbc7e7ea3e7c8b0 [term 1 LEADER]: Becoming Leader. State: Replica: e7d64c5db88940548bbc7e7ea3e7c8b0, State: Running, Role: LEADER
I20260811 05:13:34.151446 13166 consensus_queue.cc:237] T 00000000000000000000000000000000 P e7d64c5db88940548bbc7e7ea3e7c8b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d64c5db88940548bbc7e7ea3e7c8b0" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 35943 } }
I20260811 05:13:34.151856 13159 sys_catalog.cc:564] T 00000000000000000000000000000000 P e7d64c5db88940548bbc7e7ea3e7c8b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:34.153154 13168 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d64c5db88940548bbc7e7ea3e7c8b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e7d64c5db88940548bbc7e7ea3e7c8b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d64c5db88940548bbc7e7ea3e7c8b0" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 35943 } } }
I20260811 05:13:34.153280 13168 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d64c5db88940548bbc7e7ea3e7c8b0 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:34.153671 13169 sys_catalog.cc:455] T 00000000000000000000000000000000 P e7d64c5db88940548bbc7e7ea3e7c8b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e7d64c5db88940548bbc7e7ea3e7c8b0. Latest consensus state: current_term: 1 leader_uuid: "e7d64c5db88940548bbc7e7ea3e7c8b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e7d64c5db88940548bbc7e7ea3e7c8b0" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 35943 } } }
I20260811 05:13:34.153837 13169 sys_catalog.cc:458] T 00000000000000000000000000000000 P e7d64c5db88940548bbc7e7ea3e7c8b0 [sys.catalog]: This master's current role is: LEADER
W20260811 05:13:34.156464 13194 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e7d64c5db88940548bbc7e7ea3e7c8b0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:13:34.156589 13194 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:13:34.156718 13182 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:34.157413 13182 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:34.162926 13182 catalog_manager.cc:1347] Generated new cluster ID: d846385d3b28451f9c7f32973263b823
I20260811 05:13:34.162992 13182 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:34.179906 13182 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:34.181197 13182 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:34.199849 13182 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e7d64c5db88940548bbc7e7ea3e7c8b0: Generated new TSK 0
I20260811 05:13:34.200798 13182 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
I20260811 05:13:34.244453 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:35943
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:37555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:34.379520 13210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:34.379766 13210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:34.379807 13210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:34.383798 13210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:34.383857 13210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:34.383955 13210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:13:34.389331 13210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:37555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:35943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:34.390375 13210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:34.391398 13210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:34.401222 13216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:34.401237 13215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:34.401309 13221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:34.401539 13210 server_base.cc:1034] running on GCE node
I20260811 05:13:34.401939 13210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:34.402563 13210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:34.403764 13210 hybrid_clock.cc:648] HybridClock initialized: now 1786425214403749 us; error 30 us; skew 500 ppm
Aug 11 05:13:34 dist-test-slave-04bb krb5kdc[13027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425214, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:34.406870 13210 init.cc:377] Logged in from keytab as kudu/127.7.67.129@KRBTEST.COM (short username kudu)
I20260811 05:13:34.407657 13210 webserver.cc:460] Webserver started at http://127.7.67.129:35671/ using document root <none> and password file <none>
I20260811 05:13:34.408197 13210 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:34.408267 13210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:34.408510 13210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:34.410578 13210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "5b3fa1c98e5f49159c6e229c0b1b6ba5"
format_stamp: "Formatted at 2026-08-11 05:13:34 on dist-test-slave-04bb"
server_key: "b31c87a88456a141c0c2a02a76e6abf7"
server_key_iv: "d983a2ac9f9ea0bebf42ae0df299254f"
server_key_version: "encryptionkey@0"
I20260811 05:13:34.411085 13210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "5b3fa1c98e5f49159c6e229c0b1b6ba5"
format_stamp: "Formatted at 2026-08-11 05:13:34 on dist-test-slave-04bb"
server_key: "b31c87a88456a141c0c2a02a76e6abf7"
server_key_iv: "d983a2ac9f9ea0bebf42ae0df299254f"
server_key_version: "encryptionkey@0"
I20260811 05:13:34.415136 13210 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 05:13:34.417753 13240 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:34.418987 13210 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:13:34.419145 13210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "5b3fa1c98e5f49159c6e229c0b1b6ba5"
format_stamp: "Formatted at 2026-08-11 05:13:34 on dist-test-slave-04bb"
server_key: "b31c87a88456a141c0c2a02a76e6abf7"
server_key_iv: "d983a2ac9f9ea0bebf42ae0df299254f"
server_key_version: "encryptionkey@0"
I20260811 05:13:34.419263 13210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:34.435082 13210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:34.438469 13210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:34.438681 13210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:34.439296 13210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:34.440132 13210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:34.440179 13210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:34.440218 13210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:34.440243 13210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:34.449319 13210 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:43857
I20260811 05:13:34.449309 13438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:43857 every 8 connection(s)
I20260811 05:13:34.450258 13210 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:34.452198 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 13210
I20260811 05:13:34.452365 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:13:34.452750 7438 external_mini_cluster.cc:1428] Setting key 9936ad82ae7c8b6beae88a005ccc81dd
Aug 11 05:13:34 dist-test-slave-04bb krb5kdc[13027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425214, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.130@KRBTEST.COM; defaulting to no policy
I20260811 05:13:34.465698 13163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:34.452172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:56249 (local address 127.7.67.190:35943)
0811 05:13:34.452397 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:34.452402 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:34.453410 (+ 1008us) server_negotiation.cc:407] Connection header received
0811 05:13:34.455023 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:34.455030 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:34.455095 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:34.455189 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:34.456586 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.457148 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.458387 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.458644 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.458778 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:34.461553 (+ 2775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:34.461559 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:34.461565 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:34.461599 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:34.463393 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:34.463960 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:34.463963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:34.463964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:34.464047 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:34.464421 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:34.464425 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:34.464427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:34.464698 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:34.464834 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:34.465376 (+ 542us) server_negotiation.cc:299] Negotiation successful
0811 05:13:34.465517 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.130@KRBTEST.COM" created.
I20260811 05:13:34.466563 13443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:34.452716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:35943 (local address 127.7.67.129:56249)
0811 05:13:34.453251 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:34.453304 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:34.454809 (+ 1505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:34.455405 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:34.455413 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:34.455790 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:34.456370 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:34.456387 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.457301 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.457307 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:34.458218 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:34.458228 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.458817 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.458821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:34.459013 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:34.459702 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:34.459720 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:34.461367 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:34.463544 (+ 2177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:34.463549 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:34.463561 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:34.463860 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:34.464139 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:34.464141 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:34.464142 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:34.464288 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:34.464830 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:34.464836 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:34.465088 (+ 252us) client_negotiation.cc:769] Sending connection context
0811 05:13:34.465322 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 05:13:34.465651 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":163,"threads_started":1}
I20260811 05:13:34.468047 13440 heartbeater.cc:347] Connected to a master server at 127.7.67.190:35943
I20260811 05:13:34.468374 13440 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:34.469020 13440 heartbeater.cc:511] Master 127.7.67.190:35943 requested a full tablet report, sending...
I20260811 05:13:34.470721 13102 ts_manager.cc:194] Registered new tserver with Master: 5b3fa1c98e5f49159c6e229c0b1b6ba5 (127.7.67.129:43857)
I20260811 05:13:34.471927 13102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.129@KRBTEST.COM'} at 127.7.67.129:56249
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
I20260811 05:13:34.488171 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:35943
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:37555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:34.603585 13451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:34.603880 13451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:34.603960 13451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:34.607939 13451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:34.608034 13451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:34.608124 13451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:13:34.613278 13451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:37555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:35943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:34.614315 13451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:34.615336 13451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:34.627043 13459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:34.627113 13451 server_base.cc:1034] running on GCE node
W20260811 05:13:34.627086 13458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:34.627063 13465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:34.627748 13451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:34.628417 13451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:34.629608 13451 hybrid_clock.cc:648] HybridClock initialized: now 1786425214629579 us; error 41 us; skew 500 ppm
Aug 11 05:13:34 dist-test-slave-04bb krb5kdc[13027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425214, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:34.632476 13451 init.cc:377] Logged in from keytab as kudu/127.7.67.130@KRBTEST.COM (short username kudu)
I20260811 05:13:34.633419 13451 webserver.cc:460] Webserver started at http://127.7.67.130:36749/ using document root <none> and password file <none>
I20260811 05:13:34.633998 13451 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:34.634096 13451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:34.634348 13451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:34.636376 13451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "96ade429e4f345cab52540e06f4e1873"
format_stamp: "Formatted at 2026-08-11 05:13:34 on dist-test-slave-04bb"
server_key: "ab458d2ced1731b514eb115e67e29ddb"
server_key_iv: "3f858e1be2521c1a2f70045afb52d29c"
server_key_version: "encryptionkey@0"
I20260811 05:13:34.636999 13451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "96ade429e4f345cab52540e06f4e1873"
format_stamp: "Formatted at 2026-08-11 05:13:34 on dist-test-slave-04bb"
server_key: "ab458d2ced1731b514eb115e67e29ddb"
server_key_iv: "3f858e1be2521c1a2f70045afb52d29c"
server_key_version: "encryptionkey@0"
I20260811 05:13:34.640692 13451 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 05:13:34.643098 13476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:34.644678 13451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:13:34.644812 13451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "96ade429e4f345cab52540e06f4e1873"
format_stamp: "Formatted at 2026-08-11 05:13:34 on dist-test-slave-04bb"
server_key: "ab458d2ced1731b514eb115e67e29ddb"
server_key_iv: "3f858e1be2521c1a2f70045afb52d29c"
server_key_version: "encryptionkey@0"
I20260811 05:13:34.644925 13451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:34.662598 13451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:34.666075 13451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:34.666301 13451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:34.666883 13451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:34.667702 13451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:34.667788 13451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:34.667901 13451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:34.667958 13451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:34.677337 13451 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:38437
I20260811 05:13:34.677373 13659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:38437 every 8 connection(s)
I20260811 05:13:34.678212 13451 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:34.685132 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 13451
I20260811 05:13:34.685262 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:13:34.685604 7438 external_mini_cluster.cc:1428] Setting key 816fa706c73d1b9f3ec13b744dc8b7f1
Aug 11 05:13:34 dist-test-slave-04bb krb5kdc[13027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425214, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:34.692993 13163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:34.679930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:38907 (local address 127.7.67.190:35943)
0811 05:13:34.680067 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:34.680071 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:34.680906 (+ 835us) server_negotiation.cc:407] Connection header received
0811 05:13:34.682302 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:34.682306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:34.682364 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:34.682478 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:34.683615 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.684142 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.685419 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.685767 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.685894 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:34.689079 (+ 3185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:34.689086 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:34.689092 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:34.689127 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:34.690708 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:34.691330 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:34.691337 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:34.691341 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:34.691438 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:34.691799 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:34.691805 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:34.691809 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:34.692069 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:34.692178 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:34.692702 (+ 524us) server_negotiation.cc:299] Negotiation successful
0811 05:13:34.692825 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 05:13:34.693775 13662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:34.680224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:35943 (local address 127.7.67.130:38907)
0811 05:13:34.680760 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:34.680803 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:34.682081 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:34.682654 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:34.682662 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:34.683006 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:34.683465 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:34.683478 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.684282 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.684286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:34.685251 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:34.685261 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.685933 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.685936 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:34.686131 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:34.686819 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:34.686835 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:34.688905 (+ 2070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:34.690840 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:34.690847 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:34.690863 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:34.691176 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:34.691539 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:34.691541 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:34.691542 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:34.691687 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:34.692177 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:34.692182 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:34.692401 (+ 219us) client_negotiation.cc:769] Sending connection context
0811 05:13:34.692652 (+ 251us) client_negotiation.cc:240] Negotiation successful
0811 05:13:34.692966 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":110,"threads_started":1}
I20260811 05:13:34.694979 13660 heartbeater.cc:347] Connected to a master server at 127.7.67.190:35943
I20260811 05:13:34.695268 13660 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:34.695868 13660 heartbeater.cc:511] Master 127.7.67.190:35943 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.67.131@KRBTEST.COM; defaulting to no policy
I20260811 05:13:34.697202 13102 ts_manager.cc:194] Registered new tserver with Master: 96ade429e4f345cab52540e06f4e1873 (127.7.67.130:38437)
I20260811 05:13:34.697784 13102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.130@KRBTEST.COM'} at 127.7.67.130:38907
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
I20260811 05:13:34.716624 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:35943
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:37555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:34.833093 13668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:34.833355 13668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:34.833398 13668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:34.837357 13668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:34.837411 13668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:34.837468 13668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:13:34.842677 13668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:37555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:35943
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:34.843688 13668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:34.844710 13668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:34.855166 13675 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:34.855217 13677 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:34.855233 13679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:34.855765 13668 server_base.cc:1034] running on GCE node
I20260811 05:13:34.856180 13668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:34.856950 13668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:34.858142 13668 hybrid_clock.cc:648] HybridClock initialized: now 1786425214858131 us; error 29 us; skew 500 ppm
Aug 11 05:13:34 dist-test-slave-04bb krb5kdc[13027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425214, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:34.861862 13668 init.cc:377] Logged in from keytab as kudu/127.7.67.131@KRBTEST.COM (short username kudu)
I20260811 05:13:34.862934 13668 webserver.cc:460] Webserver started at http://127.7.67.131:39969/ using document root <none> and password file <none>
I20260811 05:13:34.863543 13668 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:34.863641 13668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:34.863897 13668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:34.866106 13668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "066f6add556143eca44a66f9e933a51a"
format_stamp: "Formatted at 2026-08-11 05:13:34 on dist-test-slave-04bb"
server_key: "3e25e5646df56028d097cc03fd7617db"
server_key_iv: "e232d937cc4578d2547f7da79b5707c9"
server_key_version: "encryptionkey@0"
I20260811 05:13:34.866700 13668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "066f6add556143eca44a66f9e933a51a"
format_stamp: "Formatted at 2026-08-11 05:13:34 on dist-test-slave-04bb"
server_key: "3e25e5646df56028d097cc03fd7617db"
server_key_iv: "e232d937cc4578d2547f7da79b5707c9"
server_key_version: "encryptionkey@0"
I20260811 05:13:34.870645 13668 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 05:13:34.873375 13692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:34.874724 13668 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:13:34.874871 13668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "066f6add556143eca44a66f9e933a51a"
format_stamp: "Formatted at 2026-08-11 05:13:34 on dist-test-slave-04bb"
server_key: "3e25e5646df56028d097cc03fd7617db"
server_key_iv: "e232d937cc4578d2547f7da79b5707c9"
server_key_version: "encryptionkey@0"
I20260811 05:13:34.874989 13668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:34.902912 13668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:34.906312 13668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:34.906549 13668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:34.907196 13668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:34.908133 13668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:34.908214 13668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:34.908319 13668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:34.908373 13668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:34.918135 13668 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:33009
I20260811 05:13:34.918162 13875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:33009 every 8 connection(s)
I20260811 05:13:34.919126 13668 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:13:34.923893 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 13668
I20260811 05:13:34.924041 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:13:34.924340 7438 external_mini_cluster.cc:1428] Setting key 140fcf4e47df4a02fabde629d75c3df1
Aug 11 05:13:34 dist-test-slave-04bb krb5kdc[13027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425214, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:34.933367 13163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:34.921162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:58019 (local address 127.7.67.190:35943)
0811 05:13:34.921316 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:34.921325 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:34.922109 (+ 784us) server_negotiation.cc:407] Connection header received
0811 05:13:34.923660 (+ 1551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:34.923664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:34.923727 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:34.923855 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:34.925220 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.925752 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.926903 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.927127 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.927240 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:34.929844 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:34.929847 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:34.929849 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:34.929875 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:34.931190 (+ 1315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:34.931702 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:34.931706 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:34.931708 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:34.931794 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:34.932146 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:34.932151 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:34.932153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:34.932341 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:34.932456 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:34.933075 (+ 619us) server_negotiation.cc:299] Negotiation successful
0811 05:13:34.933200 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 05:13:34.934278 13880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:34.921431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:35943 (local address 127.7.67.131:58019)
0811 05:13:34.921891 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:34.921991 (+ 100us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:34.923437 (+ 1446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:34.924054 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:34.924064 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:34.924573 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:34.925068 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:34.925082 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.925888 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.925892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:34.926770 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:34.926779 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.927288 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.927292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:34.927470 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:34.928081 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:34.928098 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:34.929655 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:34.931312 (+ 1657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:34.931318 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:34.931342 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:34.931592 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:34.931899 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:34.931903 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:34.931905 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:34.932046 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:34.932452 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:34.932459 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:34.932805 (+ 346us) client_negotiation.cc:769] Sending connection context
0811 05:13:34.933035 (+ 230us) client_negotiation.cc:240] Negotiation successful
0811 05:13:34.933367 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":112,"threads_started":1}
I20260811 05:13:34.935470 13876 heartbeater.cc:347] Connected to a master server at 127.7.67.190:35943
I20260811 05:13:34.935781 13876 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:34.936376 13876 heartbeater.cc:511] Master 127.7.67.190:35943 requested a full tablet report, sending...
I20260811 05:13:34.937582 13102 ts_manager.cc:194] Registered new tserver with Master: 066f6add556143eca44a66f9e933a51a (127.7.67.131:33009)
I20260811 05:13:34.938372 13102 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.131@KRBTEST.COM'} at 127.7.67.131:58019
I20260811 05:13:34.938380 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 05:13:34 dist-test-slave-04bb krb5kdc[13027](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 05:13:34 dist-test-slave-04bb krb5kdc[13027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.129@KRBTEST.COM
I20260811 05:13:34.967965 13888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:34.957528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41614 (local address 127.7.67.129:43857)
0811 05:13:34.957866 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:34.957886 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:34.957922 (+ 36us) server_negotiation.cc:407] Connection header received
0811 05:13:34.957979 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:34.957984 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:34.958198 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:34.958364 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:34.959068 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.959902 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.960895 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.961548 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.961649 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:34.964128 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:34.964138 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:34.964180 (+ 42us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:34.964205 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:34.966119 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:34.966539 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:34.966545 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:34.966546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:34.966629 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:34.966921 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:34.966925 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:34.966926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:34.967232 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:34.967508 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:34.967666 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 05:13:34.967803 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":152,"threads_started":1}
Aug 11 05:13:34 dist-test-slave-04bb krb5kdc[13027](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425214, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:34.979650 13163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:34.971211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53574 (local address 127.7.67.190:35943)
0811 05:13:34.971382 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:34.971386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:34.971419 (+ 33us) server_negotiation.cc:407] Connection header received
0811 05:13:34.971460 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:34.971462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:34.971504 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:34.971561 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:34.972140 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.972737 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.973652 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:34.973866 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:34.973977 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:34.976304 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:34.976308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:34.976310 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:34.976334 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:34.977825 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:34.978317 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:34.978319 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:34.978321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:34.978394 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:34.978729 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:34.978734 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:34.978740 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:34.978962 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:34.979068 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:34.979391 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 05:13:34.979484 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 05:13:34.980151 13102 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53574
I20260811 05:13:34.980880 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 13210
I20260811 05:13:34.992297 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 13451
I20260811 05:13:35.006300 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 13668
I20260811 05:13:35.019555 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 13048
2026-08-11T05:13:35Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2507 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:13:35 dist-test-slave-04bb krb5kdc[13894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:13:35 dist-test-slave-04bb krb5kdc[13894](info): set up 2 sockets
Aug 11 05:13:35 dist-test-slave-04bb krb5kdc[13894](info): commencing operation
krb5kdc: starting...
W20260811 05:13:36.283746 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:13:36 dist-test-slave-04bb krb5kdc[13894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:13:36.412271 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:46249
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:38431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:46249
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:36.525403 13917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:36.525700 13917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:36.525786 13917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:36.529872 13917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:36.529954 13917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:36.529994 13917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:36.530053 13917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:36.530093 13917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:36.535159 13917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:38431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:46249
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:46249
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:36.536204 13917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:36.537241 13917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:36.545647 13926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:36.545683 13928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:36.545647 13925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:36.546061 13917 server_base.cc:1034] running on GCE node
I20260811 05:13:36.546607 13917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:36.547605 13917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:36.548764 13917 hybrid_clock.cc:648] HybridClock initialized: now 1786425216548725 us; error 50 us; skew 500 ppm
Aug 11 05:13:36 dist-test-slave-04bb krb5kdc[13894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425216, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:36.551501 13917 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:13:36.552213 13917 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:36.552275 13917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:36.552444 13917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:36.554469 13917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "b14de403a7784e56a0ccc03c498e7c4e"
format_stamp: "Formatted at 2026-08-11 05:13:36 on dist-test-slave-04bb"
server_key: "8c0b10b00d63e791a3328c9830e62574"
server_key_iv: "4cc334165b2174d105e1cd8e9e068dea"
server_key_version: "encryptionkey@0"
I20260811 05:13:36.554991 13917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "b14de403a7784e56a0ccc03c498e7c4e"
format_stamp: "Formatted at 2026-08-11 05:13:36 on dist-test-slave-04bb"
server_key: "8c0b10b00d63e791a3328c9830e62574"
server_key_iv: "4cc334165b2174d105e1cd8e9e068dea"
server_key_version: "encryptionkey@0"
I20260811 05:13:36.558658 13917 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:13:36.561053 13935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:36.562315 13917 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:13:36.562490 13917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "b14de403a7784e56a0ccc03c498e7c4e"
format_stamp: "Formatted at 2026-08-11 05:13:36 on dist-test-slave-04bb"
server_key: "8c0b10b00d63e791a3328c9830e62574"
server_key_iv: "4cc334165b2174d105e1cd8e9e068dea"
server_key_version: "encryptionkey@0"
I20260811 05:13:36.562608 13917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:36.579080 13917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:36.582208 13917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:36.582432 13917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:36.589751 13917 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:46249
I20260811 05:13:36.590059 14017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:46249 every 8 connection(s)
I20260811 05:13:36.590453 13917 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:36.593502 14021 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:36.599141 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 13917
I20260811 05:13:36.599287 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:13:36.599602 7438 external_mini_cluster.cc:1428] Setting key a6213a9a2749cdbb8918a6b21acc0f5e
I20260811 05:13:36.599838 14021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b14de403a7784e56a0ccc03c498e7c4e: Bootstrap starting.
I20260811 05:13:36.602587 14021 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b14de403a7784e56a0ccc03c498e7c4e: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:36.603400 14021 log.cc:824] T 00000000000000000000000000000000 P b14de403a7784e56a0ccc03c498e7c4e: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:36.605459 14021 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b14de403a7784e56a0ccc03c498e7c4e: No bootstrap required, opened a new log
Aug 11 05:13:36 dist-test-slave-04bb krb5kdc[13894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:36.609622 14021 raft_consensus.cc:348] T 00000000000000000000000000000000 P b14de403a7784e56a0ccc03c498e7c4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b14de403a7784e56a0ccc03c498e7c4e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 46249 } }
I20260811 05:13:36.609927 14021 raft_consensus.cc:374] T 00000000000000000000000000000000 P b14de403a7784e56a0ccc03c498e7c4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:36.610037 14021 raft_consensus.cc:729] T 00000000000000000000000000000000 P b14de403a7784e56a0ccc03c498e7c4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b14de403a7784e56a0ccc03c498e7c4e, State: Initialized, Role: FOLLOWER
I20260811 05:13:36.610540 14021 consensus_queue.cc:260] T 00000000000000000000000000000000 P b14de403a7784e56a0ccc03c498e7c4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b14de403a7784e56a0ccc03c498e7c4e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 46249 } }
I20260811 05:13:36.610718 14021 raft_consensus.cc:388] T 00000000000000000000000000000000 P b14de403a7784e56a0ccc03c498e7c4e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:36.610832 14021 raft_consensus.cc:482] T 00000000000000000000000000000000 P b14de403a7784e56a0ccc03c498e7c4e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:36.610967 14021 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b14de403a7784e56a0ccc03c498e7c4e [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:36.611881 14021 raft_consensus.cc:504] T 00000000000000000000000000000000 P b14de403a7784e56a0ccc03c498e7c4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b14de403a7784e56a0ccc03c498e7c4e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 46249 } }
I20260811 05:13:36.612247 14021 leader_election.cc:302] T 00000000000000000000000000000000 P b14de403a7784e56a0ccc03c498e7c4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b14de403a7784e56a0ccc03c498e7c4e; no voters:
I20260811 05:13:36.612666 14021 leader_election.cc:288] T 00000000000000000000000000000000 P b14de403a7784e56a0ccc03c498e7c4e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:36.612766 14030 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b14de403a7784e56a0ccc03c498e7c4e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:36.612999 14030 raft_consensus.cc:686] T 00000000000000000000000000000000 P b14de403a7784e56a0ccc03c498e7c4e [term 1 LEADER]: Becoming Leader. State: Replica: b14de403a7784e56a0ccc03c498e7c4e, State: Running, Role: LEADER
I20260811 05:13:36.613408 14030 consensus_queue.cc:237] T 00000000000000000000000000000000 P b14de403a7784e56a0ccc03c498e7c4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b14de403a7784e56a0ccc03c498e7c4e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 46249 } }
I20260811 05:13:36.614130 14021 sys_catalog.cc:564] T 00000000000000000000000000000000 P b14de403a7784e56a0ccc03c498e7c4e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:36.614748 14030 sys_catalog.cc:455] T 00000000000000000000000000000000 P b14de403a7784e56a0ccc03c498e7c4e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b14de403a7784e56a0ccc03c498e7c4e. Latest consensus state: current_term: 1 leader_uuid: "b14de403a7784e56a0ccc03c498e7c4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b14de403a7784e56a0ccc03c498e7c4e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 46249 } } }
I20260811 05:13:36.614862 14030 sys_catalog.cc:458] T 00000000000000000000000000000000 P b14de403a7784e56a0ccc03c498e7c4e [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:36.615046 14028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:36.601135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45278 (local address 127.7.67.190:46249)
0811 05:13:36.601600 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:36.601615 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:36.601652 (+ 37us) server_negotiation.cc:407] Connection header received
0811 05:13:36.602269 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:36.602286 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:36.602611 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:36.602990 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:36.603822 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:36.604865 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:36.605821 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:36.606315 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:36.606493 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:36.609105 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:36.609110 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:36.609124 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:36.609151 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:36.611233 (+ 2082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:36.611684 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:36.611690 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:36.611695 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:36.611829 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:36.612139 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:36.612142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:36.612144 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:36.612682 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:36.613549 (+ 867us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:36.613706 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 05:13:36.614054 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":143,"threads_started":1}
I20260811 05:13:36.615551 14031 sys_catalog.cc:455] T 00000000000000000000000000000000 P b14de403a7784e56a0ccc03c498e7c4e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b14de403a7784e56a0ccc03c498e7c4e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b14de403a7784e56a0ccc03c498e7c4e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 46249 } } }
I20260811 05:13:36.615655 14031 sys_catalog.cc:458] T 00000000000000000000000000000000 P b14de403a7784e56a0ccc03c498e7c4e [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:36.615936 14037 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:36.619259 14037 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:36.625576 14037 catalog_manager.cc:1347] Generated new cluster ID: a096e49e912049d4995dbf08a2cfa6cd
I20260811 05:13:36.625640 14037 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:36.653736 14037 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:36.654790 14037 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:36.660840 14037 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b14de403a7784e56a0ccc03c498e7c4e: Generated new TSK 0
I20260811 05:13:36.661659 14037 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
I20260811 05:13:36.700922 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:46249
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:38431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:36.817714 14066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:36.818003 14066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:36.818085 14066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:36.822036 14066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:36.822136 14066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:36.822242 14066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:13:36.827510 14066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:38431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:46249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:36.828642 14066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:36.829702 14066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:36.839449 14079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:36.839489 14081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:36.839426 14076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:36.839722 14066 server_base.cc:1034] running on GCE node
I20260811 05:13:36.840224 14066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:36.840943 14066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:36.842137 14066 hybrid_clock.cc:648] HybridClock initialized: now 1786425216842106 us; error 57 us; skew 500 ppm
Aug 11 05:13:36 dist-test-slave-04bb krb5kdc[13894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425216, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:36.849051 14066 init.cc:377] Logged in from keytab as kudu/127.7.67.129@KRBTEST.COM (short username kudu)
I20260811 05:13:36.849815 14066 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:36.849881 14066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:36.850109 14066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:36.852257 14066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "d30a375071184881b109a66be5d01076"
format_stamp: "Formatted at 2026-08-11 05:13:36 on dist-test-slave-04bb"
server_key: "c405642a9509e2669ac0b740e05363f1"
server_key_iv: "10826783ee741341af7e94710c322ab5"
server_key_version: "encryptionkey@0"
I20260811 05:13:36.852898 14066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "d30a375071184881b109a66be5d01076"
format_stamp: "Formatted at 2026-08-11 05:13:36 on dist-test-slave-04bb"
server_key: "c405642a9509e2669ac0b740e05363f1"
server_key_iv: "10826783ee741341af7e94710c322ab5"
server_key_version: "encryptionkey@0"
I20260811 05:13:36.856716 14066 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 05:13:36.859302 14093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:36.860741 14066 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:13:36.860893 14066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "d30a375071184881b109a66be5d01076"
format_stamp: "Formatted at 2026-08-11 05:13:36 on dist-test-slave-04bb"
server_key: "c405642a9509e2669ac0b740e05363f1"
server_key_iv: "10826783ee741341af7e94710c322ab5"
server_key_version: "encryptionkey@0"
I20260811 05:13:36.861016 14066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:36.870936 14066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:36.874125 14066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:36.874368 14066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:36.874974 14066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:36.875837 14066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:36.875911 14066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:36.875982 14066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:36.876027 14066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:36.885529 14066 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:44555
I20260811 05:13:36.885548 14272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:44555 every 8 connection(s)
I20260811 05:13:36.886186 14066 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:36.887883 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 14066
I20260811 05:13:36.888024 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:13:36.888365 7438 external_mini_cluster.cc:1428] Setting key ee2f4e00bf23c84cb0ea9d6aca7949db
Aug 11 05:13:36 dist-test-slave-04bb krb5kdc[13894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425216, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.130@KRBTEST.COM; defaulting to no policy
I20260811 05:13:36.902477 14028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:36.888332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:49721 (local address 127.7.67.190:46249)
0811 05:13:36.888497 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:36.888501 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:36.890205 (+ 1704us) server_negotiation.cc:407] Connection header received
0811 05:13:36.891092 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:36.891099 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:36.891201 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:36.891314 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:36.892711 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:36.893254 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:36.894474 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:36.894712 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:36.894827 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:36.898051 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:36.898054 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:36.898056 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:36.898085 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:36.899821 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:36.900468 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:36.900473 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:36.900474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:36.900614 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:36.901111 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:36.901114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:36.901116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:36.901341 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:36.901463 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:36.902128 (+ 665us) server_negotiation.cc:299] Negotiation successful
0811 05:13:36.902281 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.130@KRBTEST.COM" created.
I20260811 05:13:36.903627 14281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:36.888795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:46249 (local address 127.7.67.129:49721)
0811 05:13:36.889906 (+ 1111us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:36.889999 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:36.890865 (+ 866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:36.891531 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:36.891539 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:36.891922 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:36.892491 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:36.892508 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:36.893403 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:36.893406 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:36.894331 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:36.894341 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:36.894889 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:36.894894 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:36.895075 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:36.895860 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:36.895879 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:36.897822 (+ 1943us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:36.899985 (+ 2163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:36.899992 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:36.900013 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:36.900328 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:36.900795 (+ 467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:36.900799 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:36.900801 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:36.900976 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:36.901463 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:36.901472 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:36.901771 (+ 299us) client_negotiation.cc:769] Sending connection context
0811 05:13:36.902026 (+ 255us) client_negotiation.cc:240] Negotiation successful
0811 05:13:36.902431 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":173,"threads_started":1}
I20260811 05:13:36.905179 14275 heartbeater.cc:347] Connected to a master server at 127.7.67.190:46249
I20260811 05:13:36.905551 14275 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:36.906234 14275 heartbeater.cc:511] Master 127.7.67.190:46249 requested a full tablet report, sending...
I20260811 05:13:36.908144 13962 ts_manager.cc:194] Registered new tserver with Master: d30a375071184881b109a66be5d01076 (127.7.67.129:44555)
I20260811 05:13:36.909454 13962 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.129@KRBTEST.COM'} at 127.7.67.129:49721
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
I20260811 05:13:36.923745 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:46249
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:38431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:37.045445 14288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:37.045718 14288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:37.045769 14288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:37.050319 14288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:37.050433 14288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:37.050523 14288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:13:37.056329 14288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:38431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:46249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:37.057546 14288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:37.058600 14288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:37.069061 14302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:37.069105 14297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:37.069056 14299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:37.069314 14288 server_base.cc:1034] running on GCE node
I20260811 05:13:37.069856 14288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:37.070545 14288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:37.071738 14288 hybrid_clock.cc:648] HybridClock initialized: now 1786425217071694 us; error 53 us; skew 500 ppm
Aug 11 05:13:37 dist-test-slave-04bb krb5kdc[13894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425217, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:37.074787 14288 init.cc:377] Logged in from keytab as kudu/127.7.67.130@KRBTEST.COM (short username kudu)
I20260811 05:13:37.075525 14288 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:37.075625 14288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:37.075866 14288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:37.077955 14288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "f7ade9b478474a7883708eea1689a737"
format_stamp: "Formatted at 2026-08-11 05:13:37 on dist-test-slave-04bb"
server_key: "72c133f8ad035263b7180e8531c2755b"
server_key_iv: "7a2217e197f7f37bdec5807ada6c2a3b"
server_key_version: "encryptionkey@0"
I20260811 05:13:37.078528 14288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "f7ade9b478474a7883708eea1689a737"
format_stamp: "Formatted at 2026-08-11 05:13:37 on dist-test-slave-04bb"
server_key: "72c133f8ad035263b7180e8531c2755b"
server_key_iv: "7a2217e197f7f37bdec5807ada6c2a3b"
server_key_version: "encryptionkey@0"
I20260811 05:13:37.082343 14288 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 05:13:37.084802 14315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:37.086059 14288 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:13:37.086198 14288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "f7ade9b478474a7883708eea1689a737"
format_stamp: "Formatted at 2026-08-11 05:13:37 on dist-test-slave-04bb"
server_key: "72c133f8ad035263b7180e8531c2755b"
server_key_iv: "7a2217e197f7f37bdec5807ada6c2a3b"
server_key_version: "encryptionkey@0"
I20260811 05:13:37.086338 14288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:37.110694 14288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:37.114157 14288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:37.114368 14288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:37.115042 14288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:37.116214 14288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:37.116297 14288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:37.116374 14288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:37.116427 14288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:37.125851 14288 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:37969
I20260811 05:13:37.125870 14537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:37969 every 8 connection(s)
I20260811 05:13:37.126466 14288 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:37.131861 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 14288
I20260811 05:13:37.132098 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:13:37.132444 7438 external_mini_cluster.cc:1428] Setting key 58eb19d2872978499d3224af1be85f71
Aug 11 05:13:37 dist-test-slave-04bb krb5kdc[13894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425217, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:37.141222 14028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:37.128335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:44527 (local address 127.7.67.190:46249)
0811 05:13:37.128499 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:37.128505 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:37.129535 (+ 1030us) server_negotiation.cc:407] Connection header received
0811 05:13:37.130817 (+ 1282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:37.130820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:37.130871 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:37.130954 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:37.132186 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.133016 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.134327 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.134552 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.134666 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:37.137449 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:37.137453 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:37.137455 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:37.137489 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:37.139081 (+ 1592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:37.139613 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:37.139617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:37.139619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:37.139705 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:37.140065 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:37.140069 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:37.140071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:37.140251 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:37.140361 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:37.140920 (+ 559us) server_negotiation.cc:299] Negotiation successful
0811 05:13:37.141052 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 05:13:37.142032 14544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:37.128841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:46249 (local address 127.7.67.130:44527)
0811 05:13:37.129368 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:37.129415 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:37.130585 (+ 1170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:37.131142 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:37.131149 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:37.131521 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:37.132005 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:37.132020 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.133154 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.133158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:37.134195 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:37.134206 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.134720 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.134724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:37.134896 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:37.135568 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:37.135584 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:37.137244 (+ 1660us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:37.139196 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:37.139202 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:37.139218 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:37.139496 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:37.139819 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:37.139822 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:37.139824 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:37.139965 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:37.140361 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:37.140367 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:37.140682 (+ 315us) client_negotiation.cc:769] Sending connection context
0811 05:13:37.140875 (+ 193us) client_negotiation.cc:240] Negotiation successful
0811 05:13:37.141190 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":123,"threads_started":1}
WARNING: no policy specified for kudu/127.7.67.131@KRBTEST.COM; defaulting to no policy
I20260811 05:13:37.143213 14539 heartbeater.cc:347] Connected to a master server at 127.7.67.190:46249
I20260811 05:13:37.143482 14539 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:37.143934 14539 heartbeater.cc:511] Master 127.7.67.190:46249 requested a full tablet report, sending...
I20260811 05:13:37.145022 13962 ts_manager.cc:194] Registered new tserver with Master: f7ade9b478474a7883708eea1689a737 (127.7.67.130:37969)
I20260811 05:13:37.145546 13962 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.130@KRBTEST.COM'} at 127.7.67.130:44527
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
I20260811 05:13:37.163479 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:46249
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:38431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:37.282188 14547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:37.282467 14547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:37.282575 14547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:37.286656 14547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:37.286746 14547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:37.286854 14547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:13:37.292271 14547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:38431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:46249
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:37.293393 14547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:37.294448 14547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:37.304023 14558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:37.304019 14559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:37.304023 14561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:37.304236 14547 server_base.cc:1034] running on GCE node
I20260811 05:13:37.304766 14547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:37.305382 14547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:37.306553 14547 hybrid_clock.cc:648] HybridClock initialized: now 1786425217306531 us; error 32 us; skew 500 ppm
Aug 11 05:13:37 dist-test-slave-04bb krb5kdc[13894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425217, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:37.309631 14547 init.cc:377] Logged in from keytab as kudu/127.7.67.131@KRBTEST.COM (short username kudu)
I20260811 05:13:37.310426 14547 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:37.310532 14547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:37.310791 14547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:37.313048 14547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "d83a58c17bcc497d8604a7964ba5be9a"
format_stamp: "Formatted at 2026-08-11 05:13:37 on dist-test-slave-04bb"
server_key: "832beab85466ac5aa8a86ae0e021fcf4"
server_key_iv: "c529cd114ce1dc74303b74206d947771"
server_key_version: "encryptionkey@0"
I20260811 05:13:37.313671 14547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "d83a58c17bcc497d8604a7964ba5be9a"
format_stamp: "Formatted at 2026-08-11 05:13:37 on dist-test-slave-04bb"
server_key: "832beab85466ac5aa8a86ae0e021fcf4"
server_key_iv: "c529cd114ce1dc74303b74206d947771"
server_key_version: "encryptionkey@0"
I20260811 05:13:37.318069 14547 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 05:13:37.320638 14581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:37.321834 14547 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:13:37.321974 14547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "d83a58c17bcc497d8604a7964ba5be9a"
format_stamp: "Formatted at 2026-08-11 05:13:37 on dist-test-slave-04bb"
server_key: "832beab85466ac5aa8a86ae0e021fcf4"
server_key_iv: "c529cd114ce1dc74303b74206d947771"
server_key_version: "encryptionkey@0"
I20260811 05:13:37.322085 14547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:37.337594 14547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:37.340390 14547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:37.340672 14547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:37.341235 14547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:37.342056 14547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:37.342128 14547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:37.342198 14547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:37.342250 14547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:37.351631 14547 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:33007
I20260811 05:13:37.351650 14758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:33007 every 8 connection(s)
I20260811 05:13:37.352227 14547 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:13:37.360767 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 14547
I20260811 05:13:37.360898 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:13:37.361187 7438 external_mini_cluster.cc:1428] Setting key a901c0927e4c8670828240caca0bd6de
Aug 11 05:13:37 dist-test-slave-04bb krb5kdc[13894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425217, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:37.369197 14028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:37.354018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:46151 (local address 127.7.67.190:46249)
0811 05:13:37.354154 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:37.354193 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:37.355113 (+ 920us) server_negotiation.cc:407] Connection header received
0811 05:13:37.357392 (+ 2279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:37.357396 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:37.357455 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:37.357554 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:37.359324 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.359851 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.361305 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.361609 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.361694 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:37.364916 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:37.364923 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:37.364929 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:37.364965 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:37.366851 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:37.367489 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:37.367492 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:37.367494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:37.367574 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:37.367920 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:37.367922 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:37.367924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:37.368128 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:37.368218 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:37.368889 (+ 671us) server_negotiation.cc:299] Negotiation successful
0811 05:13:37.368994 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 05:13:37.369948 14761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:37.354379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:46249 (local address 127.7.67.131:46151)
0811 05:13:37.354911 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:37.354979 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:37.357095 (+ 2116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:37.357853 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:37.357863 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:37.358611 (+ 748us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:37.359164 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:37.359179 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.359979 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.359982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:37.361151 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:37.361161 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.361794 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.361799 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:37.361991 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:37.362744 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:37.362762 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:37.364615 (+ 1853us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:37.366995 (+ 2380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:37.367001 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:37.367013 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:37.367380 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:37.367665 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:37.367667 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:37.367668 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:37.367822 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:37.368273 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:37.368280 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:37.368520 (+ 240us) client_negotiation.cc:769] Sending connection context
0811 05:13:37.368800 (+ 280us) client_negotiation.cc:240] Negotiation successful
0811 05:13:37.369067 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":112,"threads_started":1}
I20260811 05:13:37.371199 14759 heartbeater.cc:347] Connected to a master server at 127.7.67.190:46249
I20260811 05:13:37.371487 14759 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:37.372104 14759 heartbeater.cc:511] Master 127.7.67.190:46249 requested a full tablet report, sending...
I20260811 05:13:37.373647 13962 ts_manager.cc:194] Registered new tserver with Master: d83a58c17bcc497d8604a7964ba5be9a (127.7.67.131:33007)
I20260811 05:13:37.374297 13962 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.131@KRBTEST.COM'} at 127.7.67.131:46151
I20260811 05:13:37.375532 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:13:37.385301 14028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:37.377707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45284 (local address 127.7.67.190:46249)
0811 05:13:37.377857 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:37.377861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:37.378065 (+ 204us) server_negotiation.cc:407] Connection header received
0811 05:13:37.378217 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:37.378220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:37.378278 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:37.378364 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:37.379069 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.379566 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.380418 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.380696 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.380812 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:37.382104 (+ 1292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:37.382108 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:37.382110 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:37.382135 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:37.383503 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:37.383942 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:37.383946 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:37.383948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:37.384036 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:37.384324 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:37.384327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:37.384329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:37.384610 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:37.384760 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:37.385010 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 05:13:37.385126 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 05:13:37.388092 13962 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45284:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:13:37.391265 13962 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:37.406702 14774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:37.400069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:33007 (local address 127.0.0.1:44884)
0811 05:13:37.400839 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:37.400890 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:37.401020 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:37.402551 (+ 1531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:37.402556 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:37.402580 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:37.402681 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:37.402688 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.404185 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.404188 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:37.405370 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:37.405379 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.406257 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.406264 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:37.406351 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:37.406373 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 05:13:37.406455 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 05:13:37.406531 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":86,"threads_started":1}
I20260811 05:13:37.406773 14782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:37.400839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44884 (local address 127.7.67.131:33007)
0811 05:13:37.402117 (+ 1278us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:37.402128 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:37.402148 (+ 20us) server_negotiation.cc:407] Connection header received
0811 05:13:37.402217 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:37.402221 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:37.402392 (+ 171us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:37.402837 (+ 445us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:37.402991 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.404048 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.405493 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.406104 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.406205 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:37.406360 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:37.406499 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 05:13:37.406598 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1183,"thread_start_us":168,"threads_started":1}
I20260811 05:13:37.407051 14780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:37.400097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:37969 (local address 127.0.0.1:41132)
0811 05:13:37.401320 (+ 1223us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:37.401331 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:37.401404 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:37.402092 (+ 688us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:37.402099 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:37.402120 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:37.402275 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:37.402282 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.403798 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.403803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:37.405270 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:37.405282 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.406492 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.406495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:37.406572 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:37.406592 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 05:13:37.406657 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 05:13:37.406718 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1168,"mutex_wait_us":52,"thread_start_us":79,"threads_started":1}
I20260811 05:13:37.407179 14784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:37.401333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41638 (local address 127.7.67.129:44555)
0811 05:13:37.401648 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:37.401657 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:37.401852 (+ 195us) server_negotiation.cc:407] Connection header received
0811 05:13:37.401993 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:37.401998 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:37.402229 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:37.402389 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:37.402598 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.403723 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.405132 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.405891 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.406012 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:37.406160 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:37.406915 (+ 755us) server_negotiation.cc:299] Negotiation successful
0811 05:13:37.407016 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":127,"threads_started":1}
I20260811 05:13:37.407392 14783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:37.400940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41132 (local address 127.7.67.130:37969)
0811 05:13:37.401668 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:37.401676 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:37.401697 (+ 21us) server_negotiation.cc:407] Connection header received
0811 05:13:37.401771 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:37.401775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:37.401932 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:37.402102 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:37.402434 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.403314 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.405409 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.406188 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.406296 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:37.406432 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:37.407032 (+ 600us) server_negotiation.cc:299] Negotiation successful
0811 05:13:37.407152 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":126,"threads_started":1}
I20260811 05:13:37.407472 14779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:37.400436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:44555 (local address 127.0.0.1:41638)
0811 05:13:37.401738 (+ 1302us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:37.401749 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:37.401865 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:37.402388 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:37.402393 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:37.402414 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:37.402503 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:37.402509 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.403875 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.403879 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:37.405003 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:37.405013 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.406030 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.406035 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:37.406154 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:37.406217 (+ 63us) client_negotiation.cc:769] Sending connection context
0811 05:13:37.406327 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 05:13:37.407322 (+ 995us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1239,"mutex_wait_us":60,"thread_start_us":114,"threads_started":1}
I20260811 05:13:37.410214 14172 tablet_service.cc:1467] Processing CreateTablet for tablet 24d1298de38648cd979f9d00316d9fa4 (DEFAULT_TABLE table=test-table [id=381186a79f9c4cfabb04296f93510d1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:37.410346 14657 tablet_service.cc:1467] Processing CreateTablet for tablet 24d1298de38648cd979f9d00316d9fa4 (DEFAULT_TABLE table=test-table [id=381186a79f9c4cfabb04296f93510d1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:37.410347 14415 tablet_service.cc:1467] Processing CreateTablet for tablet 24d1298de38648cd979f9d00316d9fa4 (DEFAULT_TABLE table=test-table [id=381186a79f9c4cfabb04296f93510d1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:37.411424 14657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24d1298de38648cd979f9d00316d9fa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:37.411419 14415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24d1298de38648cd979f9d00316d9fa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:37.411425 14172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24d1298de38648cd979f9d00316d9fa4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:37.418664 14785 tablet_bootstrap.cc:492] T 24d1298de38648cd979f9d00316d9fa4 P d30a375071184881b109a66be5d01076: Bootstrap starting.
I20260811 05:13:37.419919 14786 tablet_bootstrap.cc:492] T 24d1298de38648cd979f9d00316d9fa4 P f7ade9b478474a7883708eea1689a737: Bootstrap starting.
I20260811 05:13:37.420099 14788 tablet_bootstrap.cc:492] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a: Bootstrap starting.
I20260811 05:13:37.421371 14785 tablet_bootstrap.cc:654] T 24d1298de38648cd979f9d00316d9fa4 P d30a375071184881b109a66be5d01076: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:37.422150 14786 tablet_bootstrap.cc:654] T 24d1298de38648cd979f9d00316d9fa4 P f7ade9b478474a7883708eea1689a737: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:37.422150 14788 tablet_bootstrap.cc:654] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:37.422456 14785 log.cc:824] T 24d1298de38648cd979f9d00316d9fa4 P d30a375071184881b109a66be5d01076: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:37.423398 14786 log.cc:824] T 24d1298de38648cd979f9d00316d9fa4 P f7ade9b478474a7883708eea1689a737: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:37.423398 14788 log.cc:824] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:37.424724 14785 tablet_bootstrap.cc:492] T 24d1298de38648cd979f9d00316d9fa4 P d30a375071184881b109a66be5d01076: No bootstrap required, opened a new log
I20260811 05:13:37.424949 14785 ts_tablet_manager.cc:1397] T 24d1298de38648cd979f9d00316d9fa4 P d30a375071184881b109a66be5d01076: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260811 05:13:37.425442 14786 tablet_bootstrap.cc:492] T 24d1298de38648cd979f9d00316d9fa4 P f7ade9b478474a7883708eea1689a737: No bootstrap required, opened a new log
I20260811 05:13:37.425441 14788 tablet_bootstrap.cc:492] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a: No bootstrap required, opened a new log
I20260811 05:13:37.425634 14786 ts_tablet_manager.cc:1397] T 24d1298de38648cd979f9d00316d9fa4 P f7ade9b478474a7883708eea1689a737: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:13:37.425639 14788 ts_tablet_manager.cc:1397] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:13:37.429282 14785 raft_consensus.cc:348] T 24d1298de38648cd979f9d00316d9fa4 P d30a375071184881b109a66be5d01076 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83a58c17bcc497d8604a7964ba5be9a" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 33007 } } peers { permanent_uuid: "f7ade9b478474a7883708eea1689a737" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 37969 } } peers { permanent_uuid: "d30a375071184881b109a66be5d01076" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 44555 } }
I20260811 05:13:37.429544 14785 raft_consensus.cc:374] T 24d1298de38648cd979f9d00316d9fa4 P d30a375071184881b109a66be5d01076 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:37.429626 14785 raft_consensus.cc:729] T 24d1298de38648cd979f9d00316d9fa4 P d30a375071184881b109a66be5d01076 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d30a375071184881b109a66be5d01076, State: Initialized, Role: FOLLOWER
I20260811 05:13:37.429486 14786 raft_consensus.cc:348] T 24d1298de38648cd979f9d00316d9fa4 P f7ade9b478474a7883708eea1689a737 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83a58c17bcc497d8604a7964ba5be9a" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 33007 } } peers { permanent_uuid: "f7ade9b478474a7883708eea1689a737" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 37969 } } peers { permanent_uuid: "d30a375071184881b109a66be5d01076" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 44555 } }
I20260811 05:13:37.429486 14788 raft_consensus.cc:348] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83a58c17bcc497d8604a7964ba5be9a" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 33007 } } peers { permanent_uuid: "f7ade9b478474a7883708eea1689a737" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 37969 } } peers { permanent_uuid: "d30a375071184881b109a66be5d01076" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 44555 } }
I20260811 05:13:37.429749 14788 raft_consensus.cc:374] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:37.429749 14786 raft_consensus.cc:374] T 24d1298de38648cd979f9d00316d9fa4 P f7ade9b478474a7883708eea1689a737 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:37.429811 14786 raft_consensus.cc:729] T 24d1298de38648cd979f9d00316d9fa4 P f7ade9b478474a7883708eea1689a737 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7ade9b478474a7883708eea1689a737, State: Initialized, Role: FOLLOWER
I20260811 05:13:37.429862 14788 raft_consensus.cc:729] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d83a58c17bcc497d8604a7964ba5be9a, State: Initialized, Role: FOLLOWER
I20260811 05:13:37.430202 14785 consensus_queue.cc:260] T 24d1298de38648cd979f9d00316d9fa4 P d30a375071184881b109a66be5d01076 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83a58c17bcc497d8604a7964ba5be9a" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 33007 } } peers { permanent_uuid: "f7ade9b478474a7883708eea1689a737" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 37969 } } peers { permanent_uuid: "d30a375071184881b109a66be5d01076" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 44555 } }
I20260811 05:13:37.430307 14786 consensus_queue.cc:260] T 24d1298de38648cd979f9d00316d9fa4 P f7ade9b478474a7883708eea1689a737 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83a58c17bcc497d8604a7964ba5be9a" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 33007 } } peers { permanent_uuid: "f7ade9b478474a7883708eea1689a737" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 37969 } } peers { permanent_uuid: "d30a375071184881b109a66be5d01076" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 44555 } }
I20260811 05:13:37.430307 14788 consensus_queue.cc:260] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83a58c17bcc497d8604a7964ba5be9a" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 33007 } } peers { permanent_uuid: "f7ade9b478474a7883708eea1689a737" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 37969 } } peers { permanent_uuid: "d30a375071184881b109a66be5d01076" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 44555 } }
I20260811 05:13:37.430902 14275 heartbeater.cc:503] Master 127.7.67.190:46249 was elected leader, sending a full tablet report...
I20260811 05:13:37.431267 14785 ts_tablet_manager.cc:1428] T 24d1298de38648cd979f9d00316d9fa4 P d30a375071184881b109a66be5d01076: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 05:13:37.431270 14788 ts_tablet_manager.cc:1428] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:13:37.431629 14759 heartbeater.cc:503] Master 127.7.67.190:46249 was elected leader, sending a full tablet report...
I20260811 05:13:37.432376 14786 ts_tablet_manager.cc:1428] T 24d1298de38648cd979f9d00316d9fa4 P f7ade9b478474a7883708eea1689a737: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 05:13:37.432746 14539 heartbeater.cc:503] Master 127.7.67.190:46249 was elected leader, sending a full tablet report...
W20260811 05:13:37.603158 14760 tablet.cc:2369] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:13:37.627556 14543 tablet.cc:2369] T 24d1298de38648cd979f9d00316d9fa4 P f7ade9b478474a7883708eea1689a737: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:13:37.637543 14278 tablet.cc:2369] T 24d1298de38648cd979f9d00316d9fa4 P d30a375071184881b109a66be5d01076: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:13:37.653065 14794 raft_consensus.cc:482] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:37.653347 14794 raft_consensus.cc:504] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83a58c17bcc497d8604a7964ba5be9a" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 33007 } } peers { permanent_uuid: "f7ade9b478474a7883708eea1689a737" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 37969 } } peers { permanent_uuid: "d30a375071184881b109a66be5d01076" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 44555 } }
I20260811 05:13:37.654621 14794 leader_election.cc:288] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f7ade9b478474a7883708eea1689a737 (127.7.67.130:37969), d30a375071184881b109a66be5d01076 (127.7.67.129:44555)
I20260811 05:13:37.658335 14783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:37.654964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:45641 (local address 127.7.67.130:37969)
0811 05:13:37.655130 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:37.655137 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:37.655260 (+ 123us) server_negotiation.cc:407] Connection header received
0811 05:13:37.655346 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:37.655350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:37.655402 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:37.655486 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:37.655772 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.656286 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.657373 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.657770 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.657896 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:37.657948 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:37.658160 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 05:13:37.658229 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 05:13:37.658335 14761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:37.654860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:37969 (local address 127.7.67.131:45641)
0811 05:13:37.655107 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:37.655125 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:37.655247 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:37.655508 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:37.655511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:37.655528 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:37.655653 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:37.655658 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.656397 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.656402 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:37.657244 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:37.657252 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.657890 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.657893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:37.657997 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:37.658024 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 05:13:37.658130 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 05:13:37.658203 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163}
I20260811 05:13:37.658573 14798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:37.654960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:44555 (local address 127.7.67.131:58249)
0811 05:13:37.655338 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:37.655349 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:37.655477 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:37.655751 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:37.655753 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:37.655768 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:37.655892 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:37.655898 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.656652 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.656657 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:37.657449 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:37.657458 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.658118 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.658122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:37.658230 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:37.658262 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 05:13:37.658365 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 05:13:37.658431 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":137,"threads_started":1}
I20260811 05:13:37.658560 14784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:37.655070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:58249 (local address 127.7.67.129:44555)
0811 05:13:37.655217 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:37.655221 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:37.655441 (+ 220us) server_negotiation.cc:407] Connection header received
0811 05:13:37.655594 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:37.655597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:37.655652 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:37.655738 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:37.656000 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.656495 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.657584 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.657991 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.658122 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:37.658176 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:37.658385 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 05:13:37.658457 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 05:13:37.659118 14452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24d1298de38648cd979f9d00316d9fa4" candidate_uuid: "d83a58c17bcc497d8604a7964ba5be9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7ade9b478474a7883708eea1689a737" is_pre_election: true
I20260811 05:13:37.659446 14452 raft_consensus.cc:2454] T 24d1298de38648cd979f9d00316d9fa4 P f7ade9b478474a7883708eea1689a737 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83a58c17bcc497d8604a7964ba5be9a in term 0.
I20260811 05:13:37.659431 14199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24d1298de38648cd979f9d00316d9fa4" candidate_uuid: "d83a58c17bcc497d8604a7964ba5be9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d30a375071184881b109a66be5d01076" is_pre_election: true
I20260811 05:13:37.659732 14199 raft_consensus.cc:2454] T 24d1298de38648cd979f9d00316d9fa4 P d30a375071184881b109a66be5d01076 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d83a58c17bcc497d8604a7964ba5be9a in term 0.
I20260811 05:13:37.660022 14584 leader_election.cc:302] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d83a58c17bcc497d8604a7964ba5be9a, f7ade9b478474a7883708eea1689a737; no voters:
I20260811 05:13:37.660305 14794 raft_consensus.cc:2781] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:37.660401 14794 raft_consensus.cc:482] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:37.660455 14794 raft_consensus.cc:3037] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:37.661536 14794 raft_consensus.cc:504] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83a58c17bcc497d8604a7964ba5be9a" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 33007 } } peers { permanent_uuid: "f7ade9b478474a7883708eea1689a737" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 37969 } } peers { permanent_uuid: "d30a375071184881b109a66be5d01076" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 44555 } }
I20260811 05:13:37.661901 14794 leader_election.cc:288] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a [CANDIDATE]: Term 1 election: Requested vote from peers f7ade9b478474a7883708eea1689a737 (127.7.67.130:37969), d30a375071184881b109a66be5d01076 (127.7.67.129:44555)
I20260811 05:13:37.662252 14452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24d1298de38648cd979f9d00316d9fa4" candidate_uuid: "d83a58c17bcc497d8604a7964ba5be9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f7ade9b478474a7883708eea1689a737"
I20260811 05:13:37.662256 14199 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24d1298de38648cd979f9d00316d9fa4" candidate_uuid: "d83a58c17bcc497d8604a7964ba5be9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d30a375071184881b109a66be5d01076"
I20260811 05:13:37.662385 14452 raft_consensus.cc:3037] T 24d1298de38648cd979f9d00316d9fa4 P f7ade9b478474a7883708eea1689a737 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:37.662395 14199 raft_consensus.cc:3037] T 24d1298de38648cd979f9d00316d9fa4 P d30a375071184881b109a66be5d01076 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:37.663408 14452 raft_consensus.cc:2454] T 24d1298de38648cd979f9d00316d9fa4 P f7ade9b478474a7883708eea1689a737 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83a58c17bcc497d8604a7964ba5be9a in term 1.
I20260811 05:13:37.663417 14199 raft_consensus.cc:2454] T 24d1298de38648cd979f9d00316d9fa4 P d30a375071184881b109a66be5d01076 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d83a58c17bcc497d8604a7964ba5be9a in term 1.
I20260811 05:13:37.663763 14585 leader_election.cc:302] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: d30a375071184881b109a66be5d01076, d83a58c17bcc497d8604a7964ba5be9a, f7ade9b478474a7883708eea1689a737; no voters:
I20260811 05:13:37.663938 14794 raft_consensus.cc:2781] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:37.664186 14794 raft_consensus.cc:686] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a [term 1 LEADER]: Becoming Leader. State: Replica: d83a58c17bcc497d8604a7964ba5be9a, State: Running, Role: LEADER
I20260811 05:13:37.664565 14794 consensus_queue.cc:237] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83a58c17bcc497d8604a7964ba5be9a" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 33007 } } peers { permanent_uuid: "f7ade9b478474a7883708eea1689a737" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 37969 } } peers { permanent_uuid: "d30a375071184881b109a66be5d01076" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 44555 } }
I20260811 05:13:37.668104 13960 catalog_manager.cc:5495] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a reported cstate change: term changed from 0 to 1, leader changed from <none> to d83a58c17bcc497d8604a7964ba5be9a (127.7.67.131). New cstate: current_term: 1 leader_uuid: "d83a58c17bcc497d8604a7964ba5be9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d83a58c17bcc497d8604a7964ba5be9a" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 33007 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f7ade9b478474a7883708eea1689a737" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 37969 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d30a375071184881b109a66be5d01076" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 44555 } health_report { overall_health: UNKNOWN } } }
I20260811 05:13:37.697944 14782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:37.694365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44894 (local address 127.7.67.131:33007)
0811 05:13:37.694501 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:37.694505 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:37.694520 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:13:37.694665 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:37.694668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:37.694721 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:37.694811 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:37.695043 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.695538 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.696363 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:37.696605 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:37.696695 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:37.697072 (+ 377us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:37.697502 (+ 430us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:37.697610 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:37.697684 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 05:13:37.697735 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:13:37.704311 14452 raft_consensus.cc:1261] T 24d1298de38648cd979f9d00316d9fa4 P f7ade9b478474a7883708eea1689a737 [term 1 FOLLOWER]: Refusing update from remote peer d83a58c17bcc497d8604a7964ba5be9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:37.704319 14199 raft_consensus.cc:1261] T 24d1298de38648cd979f9d00316d9fa4 P d30a375071184881b109a66be5d01076 [term 1 FOLLOWER]: Refusing update from remote peer d83a58c17bcc497d8604a7964ba5be9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:37.705075 14799 consensus_queue.cc:1035] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f7ade9b478474a7883708eea1689a737" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 37969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:37.705257 14794 consensus_queue.cc:1035] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d30a375071184881b109a66be5d01076" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 44555 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:37.712783 14809 mvcc.cc:205] Tried to move back new op lower bound from 7317197691709648896 to 7317197691557109760. Current Snapshot: MvccSnapshot[applied={T|T < 7317197691709648896}]
I20260811 05:13:37.722030 13960 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45284:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:37.722222 13960 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45284:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:37.724730 13960 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b14de403a7784e56a0ccc03c498e7c4e: Sending DeleteTablet for 3 replicas of tablet 24d1298de38648cd979f9d00316d9fa4
I20260811 05:13:37.725597 14415 tablet_service.cc:1514] Processing DeleteTablet for tablet 24d1298de38648cd979f9d00316d9fa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:37 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:41132
I20260811 05:13:37.725726 14657 tablet_service.cc:1514] Processing DeleteTablet for tablet 24d1298de38648cd979f9d00316d9fa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:37 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:44884
I20260811 05:13:37.726003 14172 tablet_service.cc:1514] Processing DeleteTablet for tablet 24d1298de38648cd979f9d00316d9fa4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:37 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:41638
I20260811 05:13:37.726127 14821 tablet_replica.cc:330] T 24d1298de38648cd979f9d00316d9fa4 P f7ade9b478474a7883708eea1689a737: stopping tablet replica
I20260811 05:13:37.726429 14821 raft_consensus.cc:2229] T 24d1298de38648cd979f9d00316d9fa4 P f7ade9b478474a7883708eea1689a737 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:37.726512 14825 tablet_replica.cc:330] T 24d1298de38648cd979f9d00316d9fa4 P d30a375071184881b109a66be5d01076: stopping tablet replica
I20260811 05:13:37.726754 14821 raft_consensus.cc:2258] T 24d1298de38648cd979f9d00316d9fa4 P f7ade9b478474a7883708eea1689a737 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:37.726785 14825 raft_consensus.cc:2229] T 24d1298de38648cd979f9d00316d9fa4 P d30a375071184881b109a66be5d01076 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:37.727027 14822 tablet_replica.cc:330] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a: stopping tablet replica
I20260811 05:13:37.727089 14825 raft_consensus.cc:2258] T 24d1298de38648cd979f9d00316d9fa4 P d30a375071184881b109a66be5d01076 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:37.727312 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 14066
I20260811 05:13:37.733950 14822 raft_consensus.cc:2229] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:13:37.734522 14822 raft_consensus.cc:2258] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:37.737934 14821 ts_tablet_manager.cc:1905] T 24d1298de38648cd979f9d00316d9fa4 P f7ade9b478474a7883708eea1689a737: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:37.740234 14822 ts_tablet_manager.cc:1905] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:37.741658 14821 ts_tablet_manager.cc:1918] T 24d1298de38648cd979f9d00316d9fa4 P f7ade9b478474a7883708eea1689a737: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:37.741822 14821 log.cc:1196] T 24d1298de38648cd979f9d00316d9fa4 P f7ade9b478474a7883708eea1689a737: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/24d1298de38648cd979f9d00316d9fa4
I20260811 05:13:37.742195 14821 ts_tablet_manager.cc:1939] T 24d1298de38648cd979f9d00316d9fa4 P f7ade9b478474a7883708eea1689a737: Deleting consensus metadata
I20260811 05:13:37.743562 13938 catalog_manager.cc:4841] TS f7ade9b478474a7883708eea1689a737 (127.7.67.130:37969): tablet 24d1298de38648cd979f9d00316d9fa4 (table test-table [id=381186a79f9c4cfabb04296f93510d1c]) successfully deleted
I20260811 05:13:37.744956 14822 ts_tablet_manager.cc:1918] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:37.745086 14822 log.cc:1196] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/24d1298de38648cd979f9d00316d9fa4
I20260811 05:13:37.745504 14822 ts_tablet_manager.cc:1939] T 24d1298de38648cd979f9d00316d9fa4 P d83a58c17bcc497d8604a7964ba5be9a: Deleting consensus metadata
I20260811 05:13:37.746974 13936 catalog_manager.cc:4841] TS d83a58c17bcc497d8604a7964ba5be9a (127.7.67.131:33007): tablet 24d1298de38648cd979f9d00316d9fa4 (table test-table [id=381186a79f9c4cfabb04296f93510d1c]) successfully deleted
W20260811 05:13:37.747504 13940 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.129:44555 (error 108)
I20260811 05:13:37.748579 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 14288
I20260811 05:13:37.748813 14779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:37.748387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:44555 (local address 127.0.0.1:41648)
0811 05:13:37.748589 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:37.748692 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.129:44555: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 05:13:37.755359 13940 catalog_manager.cc:4584] TS d30a375071184881b109a66be5d01076 (127.7.67.129:44555): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 24d1298de38648cd979f9d00316d9fa4: Network error: Client connection negotiation failed: client connection to 127.7.67.129:44555: connect: Connection refused (error 111)
I20260811 05:13:37.765580 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 14547
I20260811 05:13:37.787640 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 13917
2026-08-11T05:13:37Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2767 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T05:13:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:37Z Disabled control of system clock
I20260811 05:13:37.825093 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:38139
--webserver_interface=127.7.67.190
--webserver_port=0
--builtin_ntp_servers=127.7.67.148:42169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:38139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 05:13:37.940238 14830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:37.940498 14830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:37.940555 14830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:37.944564 14830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:37.944618 14830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:37.944633 14830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:37.944650 14830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:37.944665 14830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:37.949374 14830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:38139
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:38139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:37.950517 14830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:37.951920 14830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:37.960541 14844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:37.960557 14843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:37.960568 14848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:37.960871 14830 server_base.cc:1034] running on GCE node
I20260811 05:13:37.961390 14830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:37.962481 14830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:37.963645 14830 hybrid_clock.cc:648] HybridClock initialized: now 1786425217963619 us; error 39 us; skew 500 ppm
I20260811 05:13:37.965581 14830 webserver.cc:460] Webserver started at http://127.7.67.190:42771/ using document root <none> and password file <none>
I20260811 05:13:37.966171 14830 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:37.966239 14830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:37.966495 14830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:37.968597 14830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "547787f7639c406e8831e295c16c79bc"
format_stamp: "Formatted at 2026-08-11 05:13:37 on dist-test-slave-04bb"
server_key: "50da3e82abb934e9d4d41f497688d056"
server_key_iv: "3ca64ccb87719660dcfcaaa788364a80"
server_key_version: "encryptionkey@0"
I20260811 05:13:37.969188 14830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "547787f7639c406e8831e295c16c79bc"
format_stamp: "Formatted at 2026-08-11 05:13:37 on dist-test-slave-04bb"
server_key: "50da3e82abb934e9d4d41f497688d056"
server_key_iv: "3ca64ccb87719660dcfcaaa788364a80"
server_key_version: "encryptionkey@0"
I20260811 05:13:37.973055 14830 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 05:13:37.975755 14856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:37.977159 14830 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:13:37.977319 14830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "547787f7639c406e8831e295c16c79bc"
format_stamp: "Formatted at 2026-08-11 05:13:37 on dist-test-slave-04bb"
server_key: "50da3e82abb934e9d4d41f497688d056"
server_key_iv: "3ca64ccb87719660dcfcaaa788364a80"
server_key_version: "encryptionkey@0"
I20260811 05:13:37.977435 14830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:37.982051 14830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:37.982652 14830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:37.982847 14830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:37.989907 14830 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:38139
I20260811 05:13:37.989913 14934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:38139 every 8 connection(s)
I20260811 05:13:37.990882 14830 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:37.991065 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 14830
I20260811 05:13:37.991204 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:13:37.991582 7438 external_mini_cluster.cc:1428] Setting key 7af014a881931ec3fefe35635ca2fa7c
I20260811 05:13:37.994398 14935 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:37.998227 14940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:37.993055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60896 (local address 127.7.67.190:38139)
0811 05:13:37.994171 (+ 1116us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:37.994195 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:37.994238 (+ 43us) server_negotiation.cc:407] Connection header received
0811 05:13:37.995102 (+ 864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:37.995125 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:37.995569 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:37.995939 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:37.996718 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:37.996724 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:37.996739 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:37.996751 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:37.996816 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:37.996839 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:37.996990 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:37.997061 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 05:13:37.997321 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":558,"threads_started":1}
I20260811 05:13:38.001194 14935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc: Bootstrap starting.
I20260811 05:13:38.003418 14935 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:38.004117 14935 log.cc:824] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:38.006040 14935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc: No bootstrap required, opened a new log
I20260811 05:13:38.009605 14935 raft_consensus.cc:348] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "547787f7639c406e8831e295c16c79bc" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38139 } }
I20260811 05:13:38.009840 14935 raft_consensus.cc:374] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:38.009902 14935 raft_consensus.cc:729] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 547787f7639c406e8831e295c16c79bc, State: Initialized, Role: FOLLOWER
I20260811 05:13:38.010357 14935 consensus_queue.cc:260] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "547787f7639c406e8831e295c16c79bc" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38139 } }
I20260811 05:13:38.010488 14935 raft_consensus.cc:388] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:38.010569 14935 raft_consensus.cc:482] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:38.010691 14935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:38.011541 14935 raft_consensus.cc:504] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "547787f7639c406e8831e295c16c79bc" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38139 } }
I20260811 05:13:38.011891 14935 leader_election.cc:302] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 547787f7639c406e8831e295c16c79bc; no voters:
I20260811 05:13:38.012204 14935 leader_election.cc:288] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:38.012416 14946 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:38.012765 14946 raft_consensus.cc:686] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc [term 1 LEADER]: Becoming Leader. State: Replica: 547787f7639c406e8831e295c16c79bc, State: Running, Role: LEADER
I20260811 05:13:38.013111 14946 consensus_queue.cc:237] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "547787f7639c406e8831e295c16c79bc" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38139 } }
I20260811 05:13:38.013521 14935 sys_catalog.cc:564] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:38.015018 14951 sys_catalog.cc:455] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 547787f7639c406e8831e295c16c79bc. Latest consensus state: current_term: 1 leader_uuid: "547787f7639c406e8831e295c16c79bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "547787f7639c406e8831e295c16c79bc" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38139 } } }
I20260811 05:13:38.015089 14948 sys_catalog.cc:455] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "547787f7639c406e8831e295c16c79bc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "547787f7639c406e8831e295c16c79bc" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38139 } } }
I20260811 05:13:38.015223 14948 sys_catalog.cc:458] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:38.015157 14951 sys_catalog.cc:458] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc [sys.catalog]: This master's current role is: LEADER
W20260811 05:13:38.018324 14967 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:13:38.018392 14967 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:13:38.018473 14964 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:38.019197 14964 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:38.025122 14964 catalog_manager.cc:1347] Generated new cluster ID: c32c7ad28fb246e992616818afb1a5d3
I20260811 05:13:38.025195 14964 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:38.057426 14964 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:38.057765 14964 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:38.063927 14964 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc: Generated new TSK 0
I20260811 05:13:38.064672 14964 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:13:38.069926 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:38139
--builtin_ntp_servers=127.7.67.148:42169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 05:13:38.188163 14975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:38.188409 14975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:38.188447 14975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:38.192812 14975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:38.192870 14975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:38.192965 14975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:13:38.197700 14975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:38139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:38.198815 14975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:38.200263 14975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:38.210302 14993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:38.210314 14985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:38.210333 14988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:38.210716 14975 server_base.cc:1034] running on GCE node
I20260811 05:13:38.211112 14975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:38.211758 14975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:38.212960 14975 hybrid_clock.cc:648] HybridClock initialized: now 1786425218212938 us; error 33 us; skew 500 ppm
I20260811 05:13:38.215096 14975 webserver.cc:460] Webserver started at http://127.7.67.129:38655/ using document root <none> and password file <none>
I20260811 05:13:38.215736 14975 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:38.215830 14975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:38.216078 14975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:38.218261 14975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "0931b684c0c74b2285389903269811ea"
format_stamp: "Formatted at 2026-08-11 05:13:38 on dist-test-slave-04bb"
server_key: "9d0412b8d25c305bc690e60cb73a82e0"
server_key_iv: "ac1c9391b95fe3a0ae3cddfee131fa9b"
server_key_version: "encryptionkey@0"
I20260811 05:13:38.218844 14975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "0931b684c0c74b2285389903269811ea"
format_stamp: "Formatted at 2026-08-11 05:13:38 on dist-test-slave-04bb"
server_key: "9d0412b8d25c305bc690e60cb73a82e0"
server_key_iv: "ac1c9391b95fe3a0ae3cddfee131fa9b"
server_key_version: "encryptionkey@0"
I20260811 05:13:38.222621 14975 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 05:13:38.225239 15004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:38.226522 14975 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:13:38.226661 14975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "0931b684c0c74b2285389903269811ea"
format_stamp: "Formatted at 2026-08-11 05:13:38 on dist-test-slave-04bb"
server_key: "9d0412b8d25c305bc690e60cb73a82e0"
server_key_iv: "ac1c9391b95fe3a0ae3cddfee131fa9b"
server_key_version: "encryptionkey@0"
I20260811 05:13:38.226790 14975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:38.231441 14975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:38.232043 14975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:38.232227 14975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:38.232806 14975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:38.233688 14975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:38.233768 14975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:38.233830 14975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:38.233881 14975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:38.242926 14975 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:41975
I20260811 05:13:38.242970 15205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:41975 every 8 connection(s)
I20260811 05:13:38.243844 14975 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:38.247269 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 14975
I20260811 05:13:38.247417 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:13:38.247692 7438 external_mini_cluster.cc:1428] Setting key b72e3892f8761a71ecbacc269d10a8ca
I20260811 05:13:38.249796 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:38139
--builtin_ntp_servers=127.7.67.148:42169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 05:13:38.250361 14940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:38.245721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:32961 (local address 127.7.67.190:38139)
0811 05:13:38.245865 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:38.245869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:38.246904 (+ 1035us) server_negotiation.cc:407] Connection header received
0811 05:13:38.248405 (+ 1501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:38.248410 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:38.248458 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:38.248571 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:38.249785 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:38.249789 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:38.249792 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:38.249806 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:38.249832 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:38.249845 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:38.249943 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:38.250132 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 05:13:38.250181 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:13:38.251119 15209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:38.246102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38139 (local address 127.7.67.129:32961)
0811 05:13:38.246732 (+ 630us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:38.246784 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:38.248206 (+ 1422us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:38.248723 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:38.248732 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:38.248847 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:38.249516 (+ 669us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:13:38.249518 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:38.249558 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:13:38.249627 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:38.249953 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:38.249965 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:38.249974 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 05:13:38.250155 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 05:13:38.250343 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":114,"threads_started":1}
I20260811 05:13:38.252360 15206 heartbeater.cc:347] Connected to a master server at 127.7.67.190:38139
I20260811 05:13:38.252625 15206 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:38.253042 15206 heartbeater.cc:511] Master 127.7.67.190:38139 requested a full tablet report, sending...
I20260811 05:13:38.254522 14880 ts_manager.cc:194] Registered new tserver with Master: 0931b684c0c74b2285389903269811ea (127.7.67.129:41975)
W20260811 05:13:38.363859 15210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:38.364133 15210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:38.364171 15210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:38.368263 15210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:38.368310 15210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:38.368389 15210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:13:38.373652 15210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:38139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:38.374667 15210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:38.376053 15210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:38.385557 15224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:38.385579 15219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:38.385557 15220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:38.386009 15210 server_base.cc:1034] running on GCE node
I20260811 05:13:38.386451 15210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:38.387116 15210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:38.388311 15210 hybrid_clock.cc:648] HybridClock initialized: now 1786425218388301 us; error 28 us; skew 500 ppm
I20260811 05:13:38.390192 15210 webserver.cc:460] Webserver started at http://127.7.67.130:39311/ using document root <none> and password file <none>
I20260811 05:13:38.390800 15210 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:38.390893 15210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:38.391113 15210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:38.393191 15210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "21662abe575a47129269c48800bc67f5"
format_stamp: "Formatted at 2026-08-11 05:13:38 on dist-test-slave-04bb"
server_key: "e0ad90d4feb11e3631c46d26334e8176"
server_key_iv: "75d5338580cfda23fe35bdef21b3a1a2"
server_key_version: "encryptionkey@0"
I20260811 05:13:38.393760 15210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "21662abe575a47129269c48800bc67f5"
format_stamp: "Formatted at 2026-08-11 05:13:38 on dist-test-slave-04bb"
server_key: "e0ad90d4feb11e3631c46d26334e8176"
server_key_iv: "75d5338580cfda23fe35bdef21b3a1a2"
server_key_version: "encryptionkey@0"
I20260811 05:13:38.397454 15210 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:13:38.399868 15237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:38.401096 15210 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:13:38.401238 15210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "21662abe575a47129269c48800bc67f5"
format_stamp: "Formatted at 2026-08-11 05:13:38 on dist-test-slave-04bb"
server_key: "e0ad90d4feb11e3631c46d26334e8176"
server_key_iv: "75d5338580cfda23fe35bdef21b3a1a2"
server_key_version: "encryptionkey@0"
I20260811 05:13:38.401348 15210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:38.405747 15210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:38.406454 15210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:38.406631 15210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:38.407120 15210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:38.407948 15210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:38.407992 15210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:38.408028 15210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:38.408043 15210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:38.416996 15210 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:39263
I20260811 05:13:38.417022 15436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:39263 every 8 connection(s)
I20260811 05:13:38.417862 15210 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:38.423938 14940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:38.419679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:58145 (local address 127.7.67.190:38139)
0811 05:13:38.419853 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:38.419859 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:38.420951 (+ 1092us) server_negotiation.cc:407] Connection header received
0811 05:13:38.422281 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:38.422284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:38.422324 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:38.422403 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:38.423527 (+ 1124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:38.423530 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:38.423533 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:38.423545 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:38.423567 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:38.423576 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:38.423661 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:38.423791 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 05:13:38.423832 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 05:13:38.424872 15442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:38.420083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38139 (local address 127.7.67.130:58145)
0811 05:13:38.420806 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:38.420845 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:38.422052 (+ 1207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:38.422557 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:38.422563 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:38.422646 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:38.423228 (+ 582us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:13:38.423230 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:38.423271 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:13:38.423364 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:38.423660 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:38.423666 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:38.423670 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:13:38.423804 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 05:13:38.423986 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":119,"threads_started":1}
I20260811 05:13:38.425992 15440 heartbeater.cc:347] Connected to a master server at 127.7.67.190:38139
I20260811 05:13:38.426245 15440 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:38.426651 15440 heartbeater.cc:511] Master 127.7.67.190:38139 requested a full tablet report, sending...
I20260811 05:13:38.426980 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 15210
I20260811 05:13:38.427101 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:13:38.427395 7438 external_mini_cluster.cc:1428] Setting key ca87bafed49b341c1bee470c1964ab5c
I20260811 05:13:38.427852 14880 ts_manager.cc:194] Registered new tserver with Master: 21662abe575a47129269c48800bc67f5 (127.7.67.130:39263)
I20260811 05:13:38.429648 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:38139
--builtin_ntp_servers=127.7.67.148:42169
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 05:13:38.542762 15444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:38.543038 15444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:38.543107 15444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:38.547129 15444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:38.547217 15444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:38.547303 15444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:13:38.552062 15444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42169
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:38139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:38.553251 15444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:38.554642 15444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:38.564671 15456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:38.564841 15444 server_base.cc:1034] running on GCE node
W20260811 05:13:38.564703 15452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:38.564709 15454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:38.565438 15444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:38.566126 15444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:38.567353 15444 hybrid_clock.cc:648] HybridClock initialized: now 1786425218567336 us; error 28 us; skew 500 ppm
I20260811 05:13:38.569428 15444 webserver.cc:460] Webserver started at http://127.7.67.131:43793/ using document root <none> and password file <none>
I20260811 05:13:38.570070 15444 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:38.570179 15444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:38.570442 15444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:38.572736 15444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "8e6e996d08434147910074d3054d1f31"
format_stamp: "Formatted at 2026-08-11 05:13:38 on dist-test-slave-04bb"
server_key: "7fd18f7dae8bb0f2d1182a7d19c47338"
server_key_iv: "2928d6fbd34eac341fb3e074c7618004"
server_key_version: "encryptionkey@0"
I20260811 05:13:38.573357 15444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "8e6e996d08434147910074d3054d1f31"
format_stamp: "Formatted at 2026-08-11 05:13:38 on dist-test-slave-04bb"
server_key: "7fd18f7dae8bb0f2d1182a7d19c47338"
server_key_iv: "2928d6fbd34eac341fb3e074c7618004"
server_key_version: "encryptionkey@0"
I20260811 05:13:38.577306 15444 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 05:13:38.579953 15467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:38.581459 15444 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:13:38.581578 15444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "8e6e996d08434147910074d3054d1f31"
format_stamp: "Formatted at 2026-08-11 05:13:38 on dist-test-slave-04bb"
server_key: "7fd18f7dae8bb0f2d1182a7d19c47338"
server_key_iv: "2928d6fbd34eac341fb3e074c7618004"
server_key_version: "encryptionkey@0"
I20260811 05:13:38.581696 15444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:38.586280 15444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:38.586898 15444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:38.587083 15444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:38.587625 15444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:38.588593 15444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:38.588668 15444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:38.588733 15444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:38.588790 15444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:38.596963 15444 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:41189
I20260811 05:13:38.597016 15657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:41189 every 8 connection(s)
I20260811 05:13:38.597874 15444 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:13:38.603966 14940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:38.599697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:58501 (local address 127.7.67.190:38139)
0811 05:13:38.599848 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:38.599852 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:38.600951 (+ 1099us) server_negotiation.cc:407] Connection header received
0811 05:13:38.602250 (+ 1299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:38.602254 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:38.602299 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:38.602372 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:38.603483 (+ 1111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:38.603487 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:38.603489 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:38.603502 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:38.603529 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:38.603538 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:38.603636 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:38.603783 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 05:13:38.603828 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 05:13:38.604862 15660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:38.600086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38139 (local address 127.7.67.131:58501)
0811 05:13:38.600802 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:38.600850 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:38.602038 (+ 1188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:38.602528 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:38.602535 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:38.602620 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:38.603219 (+ 599us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:13:38.603221 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:38.603267 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:13:38.603332 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:38.603621 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:38.603630 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:38.603636 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 05:13:38.603821 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 05:13:38.604083 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":115,"threads_started":1}
I20260811 05:13:38.605955 15658 heartbeater.cc:347] Connected to a master server at 127.7.67.190:38139
I20260811 05:13:38.606228 15658 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:38.606550 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 15444
I20260811 05:13:38.606659 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:13:38.606712 15658 heartbeater.cc:511] Master 127.7.67.190:38139 requested a full tablet report, sending...
I20260811 05:13:38.606920 7438 external_mini_cluster.cc:1428] Setting key 55fba55784a19ad8fb32005733ee5912
I20260811 05:13:38.607890 14880 ts_manager.cc:194] Registered new tserver with Master: 8e6e996d08434147910074d3054d1f31 (127.7.67.131:41189)
I20260811 05:13:38.609155 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:13:38.613353 14940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:38.611598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60902 (local address 127.7.67.190:38139)
0811 05:13:38.611731 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:38.611734 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:38.611951 (+ 217us) server_negotiation.cc:407] Connection header received
0811 05:13:38.612043 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:38.612045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:38.612076 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:38.612140 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:38.612942 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:38.612946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:38.612948 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:38.612960 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:38.612980 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:38.612989 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:38.613088 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:38.613185 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 05:13:38.613227 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:13:38.616406 14880 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60902:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:13:38.619486 14880 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:38.631049 15673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:38.627685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:41189 (local address 127.0.0.1:47956)
0811 05:13:38.628093 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:38.628146 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:38.628259 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:38.629198 (+ 939us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:38.629204 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:38.629285 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:38.629834 (+ 549us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:13:38.629838 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:38.629860 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:13:38.629915 (+ 55us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:38.630723 (+ 808us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:38.630736 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:38.630743 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 05:13:38.630840 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 05:13:38.630929 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":111,"threads_started":1}
I20260811 05:13:38.631928 15674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:38.627881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47956 (local address 127.7.67.131:41189)
0811 05:13:38.628735 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:38.628751 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:38.628778 (+ 27us) server_negotiation.cc:407] Connection header received
0811 05:13:38.628833 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:38.628839 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:38.629029 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:38.629554 (+ 525us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:38.630503 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:38.630514 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:38.630527 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:38.630542 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:38.630582 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:38.630608 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:38.631644 (+ 1036us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:38.631689 (+ 45us) server_negotiation.cc:299] Negotiation successful
0811 05:13:38.631760 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":747,"thread_start_us":151,"threads_started":1}
I20260811 05:13:38.632325 15678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:38.628158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:41975 (local address 127.0.0.1:59940)
0811 05:13:38.629029 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:38.629043 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:38.629120 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:38.630308 (+ 1188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:38.630314 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:38.630333 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:38.631584 (+ 1251us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:13:38.631588 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:38.631602 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:13:38.631634 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:38.632046 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:38.632049 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:38.632052 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 05:13:38.632126 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 05:13:38.632163 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":804,"thread_start_us":102,"threads_started":1}
I20260811 05:13:38.632812 15680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:38.628627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59940 (local address 127.7.67.129:41975)
0811 05:13:38.629140 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:38.629153 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:38.629181 (+ 28us) server_negotiation.cc:407] Connection header received
0811 05:13:38.629278 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:38.629286 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:38.629451 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:38.629591 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:38.631780 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:38.631786 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:38.631797 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:38.631811 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:38.631855 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:38.631892 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:38.631999 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:38.632511 (+ 512us) server_negotiation.cc:299] Negotiation successful
0811 05:13:38.632636 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":129,"threads_started":1}
I20260811 05:13:38.634343 15673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:38.628466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:39263 (local address 127.0.0.1:40078)
0811 05:13:38.631231 (+ 2765us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:38.631257 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:38.631318 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:38.632028 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:38.632033 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:38.632050 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:38.632565 (+ 515us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:13:38.632567 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:38.632577 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:13:38.632602 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:38.634098 (+ 1496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:38.634101 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:38.634105 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:13:38.634196 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 05:13:38.634231 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2698,"thread_start_us":118,"threads_started":1}
I20260811 05:13:38.634433 15681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:38.630667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40078 (local address 127.7.67.130:39263)
0811 05:13:38.631574 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:38.631588 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:38.631612 (+ 24us) server_negotiation.cc:407] Connection header received
0811 05:13:38.631668 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:38.631673 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:38.631856 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:38.632252 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:38.633898 (+ 1646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:38.633903 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:38.633912 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:38.633922 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:38.633960 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:38.633982 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:38.634096 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:38.634205 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 05:13:38.634272 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":796,"thread_start_us":545,"threads_started":1}
I20260811 05:13:38.635299 15548 tablet_service.cc:1467] Processing CreateTablet for tablet 6dd8fdc7e6564e6c986f06e9f93714e7 (DEFAULT_TABLE table=test-table [id=a40e345ca6db4c718b83436445389843]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:38.635617 15088 tablet_service.cc:1467] Processing CreateTablet for tablet 6dd8fdc7e6564e6c986f06e9f93714e7 (DEFAULT_TABLE table=test-table [id=a40e345ca6db4c718b83436445389843]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:38.636399 15548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dd8fdc7e6564e6c986f06e9f93714e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:38.636586 15088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dd8fdc7e6564e6c986f06e9f93714e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:38.639237 15328 tablet_service.cc:1467] Processing CreateTablet for tablet 6dd8fdc7e6564e6c986f06e9f93714e7 (DEFAULT_TABLE table=test-table [id=a40e345ca6db4c718b83436445389843]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:38.640246 15328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dd8fdc7e6564e6c986f06e9f93714e7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:38.642716 15682 tablet_bootstrap.cc:492] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 8e6e996d08434147910074d3054d1f31: Bootstrap starting.
I20260811 05:13:38.644440 15683 tablet_bootstrap.cc:492] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea: Bootstrap starting.
I20260811 05:13:38.645179 15682 tablet_bootstrap.cc:654] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 8e6e996d08434147910074d3054d1f31: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:38.645941 15682 log.cc:824] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 8e6e996d08434147910074d3054d1f31: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:38.646652 15683 tablet_bootstrap.cc:654] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:38.647471 15683 log.cc:824] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:38.647820 15682 tablet_bootstrap.cc:492] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 8e6e996d08434147910074d3054d1f31: No bootstrap required, opened a new log
I20260811 05:13:38.648020 15685 tablet_bootstrap.cc:492] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 21662abe575a47129269c48800bc67f5: Bootstrap starting.
I20260811 05:13:38.648085 15682 ts_tablet_manager.cc:1397] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 8e6e996d08434147910074d3054d1f31: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:13:38.649430 15683 tablet_bootstrap.cc:492] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea: No bootstrap required, opened a new log
I20260811 05:13:38.649626 15683 ts_tablet_manager.cc:1397] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:13:38.650195 15685 tablet_bootstrap.cc:654] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 21662abe575a47129269c48800bc67f5: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:38.651014 15685 log.cc:824] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 21662abe575a47129269c48800bc67f5: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:38.653122 15685 tablet_bootstrap.cc:492] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 21662abe575a47129269c48800bc67f5: No bootstrap required, opened a new log
I20260811 05:13:38.653326 15685 ts_tablet_manager.cc:1397] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 21662abe575a47129269c48800bc67f5: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:13:38.653122 15682 raft_consensus.cc:348] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 8e6e996d08434147910074d3054d1f31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6e996d08434147910074d3054d1f31" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41189 } } peers { permanent_uuid: "21662abe575a47129269c48800bc67f5" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39263 } } peers { permanent_uuid: "0931b684c0c74b2285389903269811ea" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 41975 } }
I20260811 05:13:38.653402 15682 raft_consensus.cc:374] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 8e6e996d08434147910074d3054d1f31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:38.653285 15683 raft_consensus.cc:348] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6e996d08434147910074d3054d1f31" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41189 } } peers { permanent_uuid: "21662abe575a47129269c48800bc67f5" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39263 } } peers { permanent_uuid: "0931b684c0c74b2285389903269811ea" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 41975 } }
I20260811 05:13:38.653489 15682 raft_consensus.cc:729] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 8e6e996d08434147910074d3054d1f31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e6e996d08434147910074d3054d1f31, State: Initialized, Role: FOLLOWER
I20260811 05:13:38.653530 15683 raft_consensus.cc:374] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:38.653597 15683 raft_consensus.cc:729] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0931b684c0c74b2285389903269811ea, State: Initialized, Role: FOLLOWER
I20260811 05:13:38.654024 15683 consensus_queue.cc:260] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6e996d08434147910074d3054d1f31" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41189 } } peers { permanent_uuid: "21662abe575a47129269c48800bc67f5" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39263 } } peers { permanent_uuid: "0931b684c0c74b2285389903269811ea" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 41975 } }
I20260811 05:13:38.653997 15682 consensus_queue.cc:260] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 8e6e996d08434147910074d3054d1f31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6e996d08434147910074d3054d1f31" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41189 } } peers { permanent_uuid: "21662abe575a47129269c48800bc67f5" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39263 } } peers { permanent_uuid: "0931b684c0c74b2285389903269811ea" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 41975 } }
I20260811 05:13:38.654884 15206 heartbeater.cc:503] Master 127.7.67.190:38139 was elected leader, sending a full tablet report...
I20260811 05:13:38.654893 15658 heartbeater.cc:503] Master 127.7.67.190:38139 was elected leader, sending a full tablet report...
I20260811 05:13:38.655040 15682 ts_tablet_manager.cc:1428] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 8e6e996d08434147910074d3054d1f31: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 05:13:38.656152 15683 ts_tablet_manager.cc:1428] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260811 05:13:38.657577 15685 raft_consensus.cc:348] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 21662abe575a47129269c48800bc67f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6e996d08434147910074d3054d1f31" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41189 } } peers { permanent_uuid: "21662abe575a47129269c48800bc67f5" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39263 } } peers { permanent_uuid: "0931b684c0c74b2285389903269811ea" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 41975 } }
I20260811 05:13:38.657848 15685 raft_consensus.cc:374] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 21662abe575a47129269c48800bc67f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:38.657919 15685 raft_consensus.cc:729] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 21662abe575a47129269c48800bc67f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21662abe575a47129269c48800bc67f5, State: Initialized, Role: FOLLOWER
I20260811 05:13:38.658363 15685 consensus_queue.cc:260] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 21662abe575a47129269c48800bc67f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6e996d08434147910074d3054d1f31" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41189 } } peers { permanent_uuid: "21662abe575a47129269c48800bc67f5" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39263 } } peers { permanent_uuid: "0931b684c0c74b2285389903269811ea" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 41975 } }
I20260811 05:13:38.659014 15440 heartbeater.cc:503] Master 127.7.67.190:38139 was elected leader, sending a full tablet report...
I20260811 05:13:38.660590 15685 ts_tablet_manager.cc:1428] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 21662abe575a47129269c48800bc67f5: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260811 05:13:38.668787 15441 tablet.cc:2369] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 21662abe575a47129269c48800bc67f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:13:38.744957 15207 tablet.cc:2369] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:13:38.813686 15690 raft_consensus.cc:482] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:38.813897 15690 raft_consensus.cc:504] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6e996d08434147910074d3054d1f31" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41189 } } peers { permanent_uuid: "21662abe575a47129269c48800bc67f5" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39263 } } peers { permanent_uuid: "0931b684c0c74b2285389903269811ea" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 41975 } }
I20260811 05:13:38.815232 15690 leader_election.cc:288] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8e6e996d08434147910074d3054d1f31 (127.7.67.131:41189), 21662abe575a47129269c48800bc67f5 (127.7.67.130:39263)
I20260811 05:13:38.817370 15692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:38.815387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:41189 (local address 127.7.67.129:45895)
0811 05:13:38.815749 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:38.815765 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:38.815874 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:38.816188 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:38.816191 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:38.816205 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:38.816703 (+ 498us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:13:38.816704 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:38.816714 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:13:38.816742 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:38.817057 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:38.817063 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:38.817069 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 05:13:38.817160 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 05:13:38.817198 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":100,"threads_started":1}
I20260811 05:13:38.817387 15674 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:38.815545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:45895 (local address 127.7.67.131:41189)
0811 05:13:38.815724 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:38.815745 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:38.815831 (+ 86us) server_negotiation.cc:407] Connection header received
0811 05:13:38.815998 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:38.816001 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:38.816045 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:38.816150 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:38.816894 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:38.816898 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:38.816900 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:38.816912 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:38.816933 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:38.816942 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:38.817021 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:38.817181 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 05:13:38.817228 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 05:13:38.818055 15693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:38.815911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:39263 (local address 127.7.67.129:50217)
0811 05:13:38.816205 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:38.816218 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:38.816313 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:38.816853 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:38.816856 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:38.816871 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:38.817395 (+ 524us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:13:38.817397 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:38.817409 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:13:38.817440 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:38.817811 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:38.817814 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:38.817816 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 05:13:38.817889 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 05:13:38.817922 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":57,"threads_started":1}
I20260811 05:13:38.818066 15681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:38.816029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:50217 (local address 127.7.67.130:39263)
0811 05:13:38.816180 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:38.816185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:38.816300 (+ 115us) server_negotiation.cc:407] Connection header received
0811 05:13:38.816682 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:38.816685 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:38.816719 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:38.816825 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:38.817592 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:38.817596 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:38.817600 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:38.817617 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:38.817637 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:38.817646 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:38.817734 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:38.817933 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 05:13:38.817971 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:13:38.818032 15578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dd8fdc7e6564e6c986f06e9f93714e7" candidate_uuid: "0931b684c0c74b2285389903269811ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e6e996d08434147910074d3054d1f31" is_pre_election: true
I20260811 05:13:38.818375 15578 raft_consensus.cc:2454] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 8e6e996d08434147910074d3054d1f31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0931b684c0c74b2285389903269811ea in term 0.
I20260811 05:13:38.818702 15353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dd8fdc7e6564e6c986f06e9f93714e7" candidate_uuid: "0931b684c0c74b2285389903269811ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21662abe575a47129269c48800bc67f5" is_pre_election: true
I20260811 05:13:38.818917 15011 leader_election.cc:302] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0931b684c0c74b2285389903269811ea, 8e6e996d08434147910074d3054d1f31; no voters:
I20260811 05:13:38.818996 15353 raft_consensus.cc:2454] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 21662abe575a47129269c48800bc67f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0931b684c0c74b2285389903269811ea in term 0.
I20260811 05:13:38.819229 15690 raft_consensus.cc:2781] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:38.819355 15690 raft_consensus.cc:482] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:38.819419 15690 raft_consensus.cc:3037] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:38.820482 15690 raft_consensus.cc:504] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6e996d08434147910074d3054d1f31" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41189 } } peers { permanent_uuid: "21662abe575a47129269c48800bc67f5" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39263 } } peers { permanent_uuid: "0931b684c0c74b2285389903269811ea" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 41975 } }
I20260811 05:13:38.820801 15690 leader_election.cc:288] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea [CANDIDATE]: Term 1 election: Requested vote from peers 8e6e996d08434147910074d3054d1f31 (127.7.67.131:41189), 21662abe575a47129269c48800bc67f5 (127.7.67.130:39263)
I20260811 05:13:38.821283 15578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dd8fdc7e6564e6c986f06e9f93714e7" candidate_uuid: "0931b684c0c74b2285389903269811ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e6e996d08434147910074d3054d1f31"
I20260811 05:13:38.821283 15353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dd8fdc7e6564e6c986f06e9f93714e7" candidate_uuid: "0931b684c0c74b2285389903269811ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21662abe575a47129269c48800bc67f5"
I20260811 05:13:38.821427 15578 raft_consensus.cc:3037] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 8e6e996d08434147910074d3054d1f31 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:38.821427 15353 raft_consensus.cc:3037] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 21662abe575a47129269c48800bc67f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:38.822891 15578 raft_consensus.cc:2454] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 8e6e996d08434147910074d3054d1f31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0931b684c0c74b2285389903269811ea in term 1.
I20260811 05:13:38.822891 15353 raft_consensus.cc:2454] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 21662abe575a47129269c48800bc67f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0931b684c0c74b2285389903269811ea in term 1.
I20260811 05:13:38.823202 15011 leader_election.cc:302] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0931b684c0c74b2285389903269811ea, 8e6e996d08434147910074d3054d1f31; no voters:
I20260811 05:13:38.823395 15690 raft_consensus.cc:2781] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:38.823647 15690 raft_consensus.cc:686] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea [term 1 LEADER]: Becoming Leader. State: Replica: 0931b684c0c74b2285389903269811ea, State: Running, Role: LEADER
I20260811 05:13:38.824046 15690 consensus_queue.cc:237] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6e996d08434147910074d3054d1f31" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41189 } } peers { permanent_uuid: "21662abe575a47129269c48800bc67f5" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39263 } } peers { permanent_uuid: "0931b684c0c74b2285389903269811ea" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 41975 } }
I20260811 05:13:38.827589 14878 catalog_manager.cc:5495] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea reported cstate change: term changed from 0 to 1, leader changed from <none> to 0931b684c0c74b2285389903269811ea (127.7.67.129). New cstate: current_term: 1 leader_uuid: "0931b684c0c74b2285389903269811ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8e6e996d08434147910074d3054d1f31" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41189 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21662abe575a47129269c48800bc67f5" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0931b684c0c74b2285389903269811ea" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 41975 } health_report { overall_health: HEALTHY } } }
W20260811 05:13:38.848770 15659 tablet.cc:2369] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 8e6e996d08434147910074d3054d1f31: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:13:38.866122 15680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:38.864587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59948 (local address 127.7.67.129:41975)
0811 05:13:38.864742 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:38.864746 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:38.864759 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:13:38.864842 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:38.864844 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:38.864905 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:38.864980 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:38.865671 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:38.865674 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:38.865676 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:38.865699 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:38.865719 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:38.865728 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:38.865807 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:38.865896 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 05:13:38.865949 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 05:13:38.872169 15353 raft_consensus.cc:1261] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 21662abe575a47129269c48800bc67f5 [term 1 FOLLOWER]: Refusing update from remote peer 0931b684c0c74b2285389903269811ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:38.872175 15578 raft_consensus.cc:1261] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 8e6e996d08434147910074d3054d1f31 [term 1 FOLLOWER]: Refusing update from remote peer 0931b684c0c74b2285389903269811ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:38.872996 15695 consensus_queue.cc:1035] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e6e996d08434147910074d3054d1f31" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:38.873178 15690 consensus_queue.cc:1035] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "21662abe575a47129269c48800bc67f5" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:38.881392 15701 mvcc.cc:205] Tried to move back new op lower bound from 7317197696493056000 to 7317197696306700288. Current Snapshot: MvccSnapshot[applied={T|T < 7317197696493056000}]
I20260811 05:13:38.882748 15702 mvcc.cc:205] Tried to move back new op lower bound from 7317197696493056000 to 7317197696306700288. Current Snapshot: MvccSnapshot[applied={T|T < 7317197696493056000}]
I20260811 05:13:38.892477 14877 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:38.892699 14877 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:38.895257 14877 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 547787f7639c406e8831e295c16c79bc: Sending DeleteTablet for 3 replicas of tablet 6dd8fdc7e6564e6c986f06e9f93714e7
I20260811 05:13:38.896106 15548 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dd8fdc7e6564e6c986f06e9f93714e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:38 UTC) from {username='slave'} at 127.0.0.1:47956
I20260811 05:13:38.896180 15088 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dd8fdc7e6564e6c986f06e9f93714e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:38 UTC) from {username='slave'} at 127.0.0.1:59940
I20260811 05:13:38.896687 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 14975
I20260811 05:13:38.896782 15719 tablet_replica.cc:330] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea: stopping tablet replica
I20260811 05:13:38.896850 15328 tablet_service.cc:1514] Processing DeleteTablet for tablet 6dd8fdc7e6564e6c986f06e9f93714e7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:38 UTC) from {username='slave'} at 127.0.0.1:40078
I20260811 05:13:38.897073 15719 raft_consensus.cc:2229] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 0931b684c0c74b2285389903269811ea [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:13:38.908661 15718 tablet_replica.cc:330] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 8e6e996d08434147910074d3054d1f31: stopping tablet replica
I20260811 05:13:38.909322 15718 raft_consensus.cc:2229] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 8e6e996d08434147910074d3054d1f31 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:38.910620 15718 raft_consensus.cc:2258] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 8e6e996d08434147910074d3054d1f31 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:38.914608 15720 tablet_replica.cc:330] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 21662abe575a47129269c48800bc67f5: stopping tablet replica
I20260811 05:13:38.915028 15720 raft_consensus.cc:2229] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 21662abe575a47129269c48800bc67f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:38.915107 15718 ts_tablet_manager.cc:1905] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 8e6e996d08434147910074d3054d1f31: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:38.915380 15720 raft_consensus.cc:2258] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 21662abe575a47129269c48800bc67f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:38.916819 15720 ts_tablet_manager.cc:1905] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 21662abe575a47129269c48800bc67f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:38.918781 15718 ts_tablet_manager.cc:1918] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 8e6e996d08434147910074d3054d1f31: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:38.918893 15718 log.cc:1196] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 8e6e996d08434147910074d3054d1f31: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/6dd8fdc7e6564e6c986f06e9f93714e7
I20260811 05:13:38.918985 15720 ts_tablet_manager.cc:1918] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 21662abe575a47129269c48800bc67f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:38.919097 15720 log.cc:1196] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 21662abe575a47129269c48800bc67f5: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/6dd8fdc7e6564e6c986f06e9f93714e7
I20260811 05:13:38.919227 15718 ts_tablet_manager.cc:1939] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 8e6e996d08434147910074d3054d1f31: Deleting consensus metadata
I20260811 05:13:38.919515 15720 ts_tablet_manager.cc:1939] T 6dd8fdc7e6564e6c986f06e9f93714e7 P 21662abe575a47129269c48800bc67f5: Deleting consensus metadata
I20260811 05:13:38.920733 14863 catalog_manager.cc:4841] TS 21662abe575a47129269c48800bc67f5 (127.7.67.130:39263): tablet 6dd8fdc7e6564e6c986f06e9f93714e7 (table test-table [id=a40e345ca6db4c718b83436445389843]) successfully deleted
I20260811 05:13:38.920989 14863 catalog_manager.cc:4841] TS 8e6e996d08434147910074d3054d1f31 (127.7.67.131:41189): tablet 6dd8fdc7e6564e6c986f06e9f93714e7 (table test-table [id=a40e345ca6db4c718b83436445389843]) successfully deleted
W20260811 05:13:38.923058 14858 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.129:41975 (error 108)
I20260811 05:13:38.923350 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 15210
I20260811 05:13:38.928625 15673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:38.928229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:41975 (local address 127.0.0.1:59962)
0811 05:13:38.928407 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:38.928498 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.129:41975: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 05:13:38.929049 14858 catalog_manager.cc:4584] TS 0931b684c0c74b2285389903269811ea (127.7.67.129:41975): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6dd8fdc7e6564e6c986f06e9f93714e7: Network error: Client connection negotiation failed: client connection to 127.7.67.129:41975: connect: Connection refused (error 111)
I20260811 05:13:38.937217 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 15444
I20260811 05:13:38.947396 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 14830
2026-08-11T05:13:38Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1158 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 05:13:38.974695 7438 mini_oidc.cc:152] Starting JWKS server
I20260811 05:13:38.976450 7438 webserver.cc:460] Webserver started at https://127.0.0.1:33283/ using document root <none> and password file <none>
I20260811 05:13:38.976675 7438 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 05:13:38.976989 7438 webserver.cc:460] Webserver started at http://127.0.0.1:37415/ using document root <none> and password file <none>
2026-08-11T05:13:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:38Z Disabled control of system clock
I20260811 05:13:38.987741 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:40495
--webserver_interface=127.7.67.190
--webserver_port=0
--builtin_ntp_servers=127.7.67.148:33861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:40495
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33283/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425190664461-7438-0/testchainca.cert with env {}
W20260811 05:13:39.122344 15737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:39.122581 15737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:39.122620 15737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:39.126642 15737 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:39.126693 15737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:39.126708 15737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:39.126724 15737 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:39.126739 15737 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 05:13:39.126754 15737 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 05:13:39.126768 15737 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33283/jwks.json
I20260811 05:13:39.131846 15737 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:33861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:40495
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:40495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33283/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425190664461-7438-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:39.132793 15737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:39.133864 15737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:39.142396 15750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:39.142396 15755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:39.142632 15737 server_base.cc:1034] running on GCE node
W20260811 05:13:39.142398 15748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:39.143272 15737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:39.144368 15737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:39.145551 15737 hybrid_clock.cc:648] HybridClock initialized: now 1786425219145533 us; error 42 us; skew 500 ppm
I20260811 05:13:39.147416 15737 webserver.cc:460] Webserver started at http://127.7.67.190:43971/ using document root <none> and password file <none>
I20260811 05:13:39.148010 15737 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:39.148103 15737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:39.148342 15737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:39.150432 15737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "e450489e25184343a4715e635f5b6a55"
format_stamp: "Formatted at 2026-08-11 05:13:39 on dist-test-slave-04bb"
server_key: "ff5dc0ec7c4550157833a42b75a7af8e"
server_key_iv: "240c7aa3194da655a6410b4d906ddbb7"
server_key_version: "encryptionkey@0"
I20260811 05:13:39.151003 15737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "e450489e25184343a4715e635f5b6a55"
format_stamp: "Formatted at 2026-08-11 05:13:39 on dist-test-slave-04bb"
server_key: "ff5dc0ec7c4550157833a42b75a7af8e"
server_key_iv: "240c7aa3194da655a6410b4d906ddbb7"
server_key_version: "encryptionkey@0"
I20260811 05:13:39.154803 15737 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260811 05:13:39.157349 15762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:39.158566 15737 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:13:39.158708 15737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "e450489e25184343a4715e635f5b6a55"
format_stamp: "Formatted at 2026-08-11 05:13:39 on dist-test-slave-04bb"
server_key: "ff5dc0ec7c4550157833a42b75a7af8e"
server_key_iv: "240c7aa3194da655a6410b4d906ddbb7"
server_key_version: "encryptionkey@0"
I20260811 05:13:39.158820 15737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:39.177462 15737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:39.178133 15737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:39.178339 15737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:39.185654 15737 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:40495
I20260811 05:13:39.185660 15862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:40495 every 8 connection(s)
I20260811 05:13:39.187026 15737 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:39.189994 15865 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:39.194482 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 15737
I20260811 05:13:39.194629 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:13:39.194902 7438 external_mini_cluster.cc:1428] Setting key d577eac6566f7a3f52198e015f8d85a4
I20260811 05:13:39.195956 15865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e450489e25184343a4715e635f5b6a55: Bootstrap starting.
I20260811 05:13:39.198796 15865 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e450489e25184343a4715e635f5b6a55: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:39.199718 15865 log.cc:824] T 00000000000000000000000000000000 P e450489e25184343a4715e635f5b6a55: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:39.201797 15865 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e450489e25184343a4715e635f5b6a55: No bootstrap required, opened a new log
I20260811 05:13:39.205919 15870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:39.196317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33162 (local address 127.7.67.190:40495)
0811 05:13:39.197028 (+ 711us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:39.197050 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:39.197100 (+ 50us) server_negotiation.cc:407] Connection header received
0811 05:13:39.197960 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:39.197984 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:39.198492 (+ 508us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:39.198874 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:39.199388 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.200763 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.201688 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.202369 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.202535 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:39.203502 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:39.203508 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:39.203525 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:39.204112 (+ 587us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:39.204183 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:39.204210 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:39.204313 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:39.204508 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 05:13:39.204825 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":153,"threads_started":1}
I20260811 05:13:39.206534 15865 raft_consensus.cc:348] T 00000000000000000000000000000000 P e450489e25184343a4715e635f5b6a55 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e450489e25184343a4715e635f5b6a55" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 40495 } }
I20260811 05:13:39.206743 15865 raft_consensus.cc:374] T 00000000000000000000000000000000 P e450489e25184343a4715e635f5b6a55 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:39.206807 15865 raft_consensus.cc:729] T 00000000000000000000000000000000 P e450489e25184343a4715e635f5b6a55 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e450489e25184343a4715e635f5b6a55, State: Initialized, Role: FOLLOWER
I20260811 05:13:39.207271 15865 consensus_queue.cc:260] T 00000000000000000000000000000000 P e450489e25184343a4715e635f5b6a55 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e450489e25184343a4715e635f5b6a55" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 40495 } }
I20260811 05:13:39.207434 15865 raft_consensus.cc:388] T 00000000000000000000000000000000 P e450489e25184343a4715e635f5b6a55 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:39.207520 15865 raft_consensus.cc:482] T 00000000000000000000000000000000 P e450489e25184343a4715e635f5b6a55 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:39.207647 15865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e450489e25184343a4715e635f5b6a55 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:39.208778 15865 raft_consensus.cc:504] T 00000000000000000000000000000000 P e450489e25184343a4715e635f5b6a55 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e450489e25184343a4715e635f5b6a55" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 40495 } }
I20260811 05:13:39.209075 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:13:39.209167 15865 leader_election.cc:302] T 00000000000000000000000000000000 P e450489e25184343a4715e635f5b6a55 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e450489e25184343a4715e635f5b6a55; no voters:
I20260811 05:13:39.209494 15865 leader_election.cc:288] T 00000000000000000000000000000000 P e450489e25184343a4715e635f5b6a55 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:39.209618 15874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e450489e25184343a4715e635f5b6a55 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:39.209882 15874 raft_consensus.cc:686] T 00000000000000000000000000000000 P e450489e25184343a4715e635f5b6a55 [term 1 LEADER]: Becoming Leader. State: Replica: e450489e25184343a4715e635f5b6a55, State: Running, Role: LEADER
I20260811 05:13:39.210228 15874 consensus_queue.cc:237] T 00000000000000000000000000000000 P e450489e25184343a4715e635f5b6a55 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e450489e25184343a4715e635f5b6a55" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 40495 } }
I20260811 05:13:39.210934 15865 sys_catalog.cc:564] T 00000000000000000000000000000000 P e450489e25184343a4715e635f5b6a55 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:39.212024 15877 sys_catalog.cc:455] T 00000000000000000000000000000000 P e450489e25184343a4715e635f5b6a55 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e450489e25184343a4715e635f5b6a55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e450489e25184343a4715e635f5b6a55" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 40495 } } }
I20260811 05:13:39.212165 15877 sys_catalog.cc:458] T 00000000000000000000000000000000 P e450489e25184343a4715e635f5b6a55 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:39.212435 15875 sys_catalog.cc:455] T 00000000000000000000000000000000 P e450489e25184343a4715e635f5b6a55 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e450489e25184343a4715e635f5b6a55. Latest consensus state: current_term: 1 leader_uuid: "e450489e25184343a4715e635f5b6a55" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e450489e25184343a4715e635f5b6a55" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 40495 } } }
I20260811 05:13:39.212553 15875 sys_catalog.cc:458] T 00000000000000000000000000000000 P e450489e25184343a4715e635f5b6a55 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:39.212654 15890 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:39.215548 15890 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:39.221088 15890 catalog_manager.cc:1347] Generated new cluster ID: 60af9137ec874065b63d6164d1896820
I20260811 05:13:39.221158 15890 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:39.227659 15890 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:39.228509 15890 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:39.233305 15890 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e450489e25184343a4715e635f5b6a55: Generated new TSK 0
I20260811 05:13:39.234030 15890 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 05:13:39.268945 15920 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 05:13:39.269122 15920 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 05:13:39.269656 15870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:39.250497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33178 (local address 127.7.67.190:40495)
0811 05:13:39.250644 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:39.250649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:39.250853 (+ 204us) server_negotiation.cc:407] Connection header received
0811 05:13:39.251063 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:39.251065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:39.251110 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:39.251202 (+ 92us) server_negotiation.cc:226] Negotiated authn=JWT
0811 05:13:39.251547 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.252077 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.253338 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.253552 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.253629 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:39.254053 (+ 424us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 05:13:39.269322 (+ 15269us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33283/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 05:13:39.269548 (+ 226us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33178: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33283/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 05:13:39.269757 15870 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33178: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33283/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 05:13:39.271267 15919 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:39.250412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:40495 (local address 127.0.0.1:33178)
0811 05:13:39.250764 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:39.250779 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:39.250915 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:39.251242 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:39.251246 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:39.251300 (+ 54us) client_negotiation.cc:189] Negotiated authn=JWT
0811 05:13:39.251437 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:39.251444 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.252193 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.252197 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:39.253216 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:39.253227 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.253670 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.253674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:39.253772 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:39.253826 (+ 54us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 05:13:39.269548 (+ 15722us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33283/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 05:13:39.269797 (+ 249us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.67.190:40495: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33283/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":107,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.7.67.190:40495: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33283/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 05:13:39.272383 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 15737
2026-08-11T05:13:39Z chronyd exiting
I20260811 05:13:39.316251 7438 test_util.cc:175] -----------------------------------------------
I20260811 05:13:39.316342 7438 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786425190664461-7438-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (343 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 05:13:39.317507 7438 mini_oidc.cc:152] Starting JWKS server
I20260811 05:13:39.318329 7438 webserver.cc:460] Webserver started at https://127.0.0.1:46183/ using document root <none> and password file <none>
I20260811 05:13:39.318406 7438 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 05:13:39.318612 7438 webserver.cc:460] Webserver started at http://127.0.0.1:45019/ using document root <none> and password file <none>
2026-08-11T05:13:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:39Z Disabled control of system clock
I20260811 05:13:39.328749 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:45225
--webserver_interface=127.7.67.190
--webserver_port=0
--builtin_ntp_servers=127.7.67.148:39621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:45225
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46183/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425190664461-7438-0/testchainca.cert with env {}
W20260811 05:13:39.445328 15927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:39.445668 15927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:39.445756 15927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:39.449793 15927 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:39.449868 15927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:39.449898 15927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:39.449934 15927 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:39.449967 15927 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 05:13:39.450002 15927 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 05:13:39.450031 15927 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46183/jwks.json
I20260811 05:13:39.455443 15927 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:39621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:45225
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:45225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46183/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425190664461-7438-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:39.456470 15927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:39.457492 15927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:39.466140 15944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:39.466184 15947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:39.466372 15927 server_base.cc:1034] running on GCE node
W20260811 05:13:39.466142 15942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:39.466943 15927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:39.468021 15927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:39.469224 15927 hybrid_clock.cc:648] HybridClock initialized: now 1786425219469203 us; error 31 us; skew 500 ppm
I20260811 05:13:39.471146 15927 webserver.cc:460] Webserver started at http://127.7.67.190:41517/ using document root <none> and password file <none>
I20260811 05:13:39.471720 15927 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:39.471813 15927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:39.472043 15927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:39.474277 15927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "b9293036b42547cdab6b8b914f12313b"
format_stamp: "Formatted at 2026-08-11 05:13:39 on dist-test-slave-04bb"
server_key: "ff5c22b513a98086a274a68d066bc694"
server_key_iv: "f6deb7c84614ac2cccc6555b63d289c5"
server_key_version: "encryptionkey@0"
I20260811 05:13:39.474834 15927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "b9293036b42547cdab6b8b914f12313b"
format_stamp: "Formatted at 2026-08-11 05:13:39 on dist-test-slave-04bb"
server_key: "ff5c22b513a98086a274a68d066bc694"
server_key_iv: "f6deb7c84614ac2cccc6555b63d289c5"
server_key_version: "encryptionkey@0"
I20260811 05:13:39.478648 15927 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 05:13:39.481057 15956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:39.482280 15927 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:13:39.482429 15927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "b9293036b42547cdab6b8b914f12313b"
format_stamp: "Formatted at 2026-08-11 05:13:39 on dist-test-slave-04bb"
server_key: "ff5c22b513a98086a274a68d066bc694"
server_key_iv: "f6deb7c84614ac2cccc6555b63d289c5"
server_key_version: "encryptionkey@0"
I20260811 05:13:39.482540 15927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:39.498368 15927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:39.499010 15927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:39.499217 15927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:39.506441 15927 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:45225
I20260811 05:13:39.506453 16039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:45225 every 8 connection(s)
I20260811 05:13:39.507387 15927 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:39.510526 16042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:39.514418 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 15927
I20260811 05:13:39.514568 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:13:39.514849 7438 external_mini_cluster.cc:1428] Setting key d576089f3983aaac885e8ca72c41ecbe
I20260811 05:13:39.516438 16042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9293036b42547cdab6b8b914f12313b: Bootstrap starting.
I20260811 05:13:39.518736 16042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b9293036b42547cdab6b8b914f12313b: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:39.519517 16042 log.cc:824] T 00000000000000000000000000000000 P b9293036b42547cdab6b8b914f12313b: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:39.521412 16042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b9293036b42547cdab6b8b914f12313b: No bootstrap required, opened a new log
I20260811 05:13:39.524319 16048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:39.516192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40678 (local address 127.7.67.190:45225)
0811 05:13:39.516942 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:39.516959 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:39.516996 (+ 37us) server_negotiation.cc:407] Connection header received
0811 05:13:39.517634 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:39.517651 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:39.518068 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:39.518459 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:39.518982 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.520007 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.520995 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.521500 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.521687 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:39.522583 (+ 896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:39.522596 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:39.522609 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:39.522986 (+ 377us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:39.523075 (+ 89us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:39.523104 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:39.523227 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:39.523347 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 05:13:39.523610 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":144,"threads_started":1}
I20260811 05:13:39.525133 16042 raft_consensus.cc:348] T 00000000000000000000000000000000 P b9293036b42547cdab6b8b914f12313b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9293036b42547cdab6b8b914f12313b" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45225 } }
I20260811 05:13:39.525338 16042 raft_consensus.cc:374] T 00000000000000000000000000000000 P b9293036b42547cdab6b8b914f12313b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:39.525417 16042 raft_consensus.cc:729] T 00000000000000000000000000000000 P b9293036b42547cdab6b8b914f12313b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9293036b42547cdab6b8b914f12313b, State: Initialized, Role: FOLLOWER
I20260811 05:13:39.525872 16042 consensus_queue.cc:260] T 00000000000000000000000000000000 P b9293036b42547cdab6b8b914f12313b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9293036b42547cdab6b8b914f12313b" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45225 } }
I20260811 05:13:39.525991 16042 raft_consensus.cc:388] T 00000000000000000000000000000000 P b9293036b42547cdab6b8b914f12313b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:39.526139 16042 raft_consensus.cc:482] T 00000000000000000000000000000000 P b9293036b42547cdab6b8b914f12313b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:39.526259 16042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b9293036b42547cdab6b8b914f12313b [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:39.527294 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:13:39.527354 16042 raft_consensus.cc:504] T 00000000000000000000000000000000 P b9293036b42547cdab6b8b914f12313b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9293036b42547cdab6b8b914f12313b" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45225 } }
I20260811 05:13:39.527678 16042 leader_election.cc:302] T 00000000000000000000000000000000 P b9293036b42547cdab6b8b914f12313b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b9293036b42547cdab6b8b914f12313b; no voters:
I20260811 05:13:39.528044 16042 leader_election.cc:288] T 00000000000000000000000000000000 P b9293036b42547cdab6b8b914f12313b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:39.528156 16051 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b9293036b42547cdab6b8b914f12313b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:39.528434 16051 raft_consensus.cc:686] T 00000000000000000000000000000000 P b9293036b42547cdab6b8b914f12313b [term 1 LEADER]: Becoming Leader. State: Replica: b9293036b42547cdab6b8b914f12313b, State: Running, Role: LEADER
I20260811 05:13:39.528831 16051 consensus_queue.cc:237] T 00000000000000000000000000000000 P b9293036b42547cdab6b8b914f12313b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9293036b42547cdab6b8b914f12313b" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45225 } }
I20260811 05:13:39.529323 16042 sys_catalog.cc:564] T 00000000000000000000000000000000 P b9293036b42547cdab6b8b914f12313b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:39.530516 16053 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9293036b42547cdab6b8b914f12313b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b9293036b42547cdab6b8b914f12313b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9293036b42547cdab6b8b914f12313b" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45225 } } }
I20260811 05:13:39.530661 16053 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9293036b42547cdab6b8b914f12313b [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:39.531047 16054 sys_catalog.cc:455] T 00000000000000000000000000000000 P b9293036b42547cdab6b8b914f12313b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b9293036b42547cdab6b8b914f12313b. Latest consensus state: current_term: 1 leader_uuid: "b9293036b42547cdab6b8b914f12313b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b9293036b42547cdab6b8b914f12313b" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45225 } } }
I20260811 05:13:39.531188 16054 sys_catalog.cc:458] T 00000000000000000000000000000000 P b9293036b42547cdab6b8b914f12313b [sys.catalog]: This master's current role is: LEADER
W20260811 05:13:39.533988 16071 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b9293036b42547cdab6b8b914f12313b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:13:39.534054 16071 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:13:39.534112 16064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:39.534852 16064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:39.539762 16064 catalog_manager.cc:1347] Generated new cluster ID: fc68608590c74e7f9c60dc077721fa35
I20260811 05:13:39.539826 16064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:39.544268 16064 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:39.545172 16064 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:39.548699 16064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b9293036b42547cdab6b8b914f12313b: Generated new TSK 0
I20260811 05:13:39.549324 16064 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 05:13:39.574744 16090 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 05:13:39.574852 16090 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 05:13:39.575506 16048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:39.559082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40694 (local address 127.7.67.190:45225)
0811 05:13:39.559223 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:39.559228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:39.559379 (+ 151us) server_negotiation.cc:407] Connection header received
0811 05:13:39.559505 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:39.559509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:39.559564 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:39.559675 (+ 111us) server_negotiation.cc:226] Negotiated authn=JWT
0811 05:13:39.559905 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.561185 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.562187 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.562549 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.562652 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:39.562923 (+ 271us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 05:13:39.575101 (+ 12178us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46183/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 05:13:39.575369 (+ 268us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40694: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46183/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 05:13:39.575572 16048 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40694: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46183/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 05:13:39.575594 16088 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:39.558964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45225 (local address 127.0.0.1:40694)
0811 05:13:39.559295 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:39.559307 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:39.559402 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:39.559688 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:39.559691 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:39.559698 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0811 05:13:39.559800 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:39.559804 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.561328 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.561331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:39.562073 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:39.562081 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.562663 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.562666 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:39.562733 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:39.562756 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 05:13:39.575304 (+ 12548us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46183/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 05:13:39.575435 (+ 131us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.67.190:45225: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46183/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":108,"threads_started":1}
I20260811 05:13:39.576318 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 15927
2026-08-11T05:13:39Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (311 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 05:13:39.628369 7438 mini_oidc.cc:152] Starting JWKS server
I20260811 05:13:39.629289 7438 webserver.cc:460] Webserver started at https://127.0.0.1:34395/ using document root <none> and password file <none>
I20260811 05:13:39.629374 7438 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 05:13:39.629580 7438 webserver.cc:460] Webserver started at http://127.0.0.1:36457/ using document root <none> and password file <none>
2026-08-11T05:13:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:39Z Disabled control of system clock
I20260811 05:13:39.640254 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:45231
--webserver_interface=127.7.67.190
--webserver_port=0
--builtin_ntp_servers=127.7.67.148:37079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:45231
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34395/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 05:13:39.753317 16104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:39.753562 16104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:39.753599 16104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:39.757520 16104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:39.757572 16104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:39.757588 16104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:39.757604 16104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:39.757619 16104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 05:13:39.757634 16104 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 05:13:39.757648 16104 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34395/jwks.json
I20260811 05:13:39.762463 16104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:37079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:45231
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:45231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34395/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:39.763410 16104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:39.764750 16104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:39.773150 16116 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:39.773355 16104 server_base.cc:1034] running on GCE node
W20260811 05:13:39.773186 16111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:39.773164 16112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:39.774044 16104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:39.775120 16104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:39.776337 16104 hybrid_clock.cc:648] HybridClock initialized: now 1786425219776323 us; error 28 us; skew 500 ppm
I20260811 05:13:39.778402 16104 webserver.cc:460] Webserver started at http://127.7.67.190:41979/ using document root <none> and password file <none>
I20260811 05:13:39.779031 16104 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:39.779098 16104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:39.779336 16104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:39.781488 16104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "cbfc33b15af8455191f1da2e10d7901f"
format_stamp: "Formatted at 2026-08-11 05:13:39 on dist-test-slave-04bb"
server_key: "1049c7c8d609b55754bee8da992e8af6"
server_key_iv: "5c89e629136ecb158f05b3a239042318"
server_key_version: "encryptionkey@0"
I20260811 05:13:39.782058 16104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "cbfc33b15af8455191f1da2e10d7901f"
format_stamp: "Formatted at 2026-08-11 05:13:39 on dist-test-slave-04bb"
server_key: "1049c7c8d609b55754bee8da992e8af6"
server_key_iv: "5c89e629136ecb158f05b3a239042318"
server_key_version: "encryptionkey@0"
I20260811 05:13:39.785872 16104 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 05:13:39.788214 16127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:39.789474 16104 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:13:39.789616 16104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "cbfc33b15af8455191f1da2e10d7901f"
format_stamp: "Formatted at 2026-08-11 05:13:39 on dist-test-slave-04bb"
server_key: "1049c7c8d609b55754bee8da992e8af6"
server_key_iv: "5c89e629136ecb158f05b3a239042318"
server_key_version: "encryptionkey@0"
I20260811 05:13:39.789719 16104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:39.825088 16104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:39.825894 16104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:39.826133 16104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:39.833889 16104 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:45231
I20260811 05:13:39.833900 16207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:45231 every 8 connection(s)
I20260811 05:13:39.834867 16104 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:39.836280 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 16104
I20260811 05:13:39.836414 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:13:39.836738 7438 external_mini_cluster.cc:1428] Setting key 3a63ede2fc239f7d7e94c2f0b304a0dc
I20260811 05:13:39.838773 16208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:39.845448 16208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbfc33b15af8455191f1da2e10d7901f: Bootstrap starting.
I20260811 05:13:39.846010 16211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:39.838049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49134 (local address 127.7.67.190:45231)
0811 05:13:39.838670 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:39.838685 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:39.838721 (+ 36us) server_negotiation.cc:407] Connection header received
0811 05:13:39.839429 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:39.839448 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:39.839867 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:39.840148 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:39.840480 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.841481 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.842680 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.843143 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.843259 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:39.844178 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:39.844183 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:39.844195 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:39.844640 (+ 445us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:39.844701 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:39.844723 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:39.844808 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:39.844980 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 05:13:39.845270 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":152,"threads_started":1}
I20260811 05:13:39.847891 16208 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cbfc33b15af8455191f1da2e10d7901f: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:39.848825 16208 log.cc:824] T 00000000000000000000000000000000 P cbfc33b15af8455191f1da2e10d7901f: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:39.849018 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:13:39.850972 16208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cbfc33b15af8455191f1da2e10d7901f: No bootstrap required, opened a new log
I20260811 05:13:39.854457 16208 raft_consensus.cc:348] T 00000000000000000000000000000000 P cbfc33b15af8455191f1da2e10d7901f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbfc33b15af8455191f1da2e10d7901f" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45231 } }
I20260811 05:13:39.854688 16208 raft_consensus.cc:374] T 00000000000000000000000000000000 P cbfc33b15af8455191f1da2e10d7901f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:39.854758 16208 raft_consensus.cc:729] T 00000000000000000000000000000000 P cbfc33b15af8455191f1da2e10d7901f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cbfc33b15af8455191f1da2e10d7901f, State: Initialized, Role: FOLLOWER
I20260811 05:13:39.855218 16208 consensus_queue.cc:260] T 00000000000000000000000000000000 P cbfc33b15af8455191f1da2e10d7901f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbfc33b15af8455191f1da2e10d7901f" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45231 } }
I20260811 05:13:39.855379 16208 raft_consensus.cc:388] T 00000000000000000000000000000000 P cbfc33b15af8455191f1da2e10d7901f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:39.855470 16208 raft_consensus.cc:482] T 00000000000000000000000000000000 P cbfc33b15af8455191f1da2e10d7901f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:39.855590 16208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cbfc33b15af8455191f1da2e10d7901f [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:39.856626 16208 raft_consensus.cc:504] T 00000000000000000000000000000000 P cbfc33b15af8455191f1da2e10d7901f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbfc33b15af8455191f1da2e10d7901f" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45231 } }
I20260811 05:13:39.856987 16208 leader_election.cc:302] T 00000000000000000000000000000000 P cbfc33b15af8455191f1da2e10d7901f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cbfc33b15af8455191f1da2e10d7901f; no voters:
I20260811 05:13:39.857311 16208 leader_election.cc:288] T 00000000000000000000000000000000 P cbfc33b15af8455191f1da2e10d7901f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:39.857451 16215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cbfc33b15af8455191f1da2e10d7901f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:39.857731 16215 raft_consensus.cc:686] T 00000000000000000000000000000000 P cbfc33b15af8455191f1da2e10d7901f [term 1 LEADER]: Becoming Leader. State: Replica: cbfc33b15af8455191f1da2e10d7901f, State: Running, Role: LEADER
I20260811 05:13:39.858038 16215 consensus_queue.cc:237] T 00000000000000000000000000000000 P cbfc33b15af8455191f1da2e10d7901f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbfc33b15af8455191f1da2e10d7901f" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45231 } }
I20260811 05:13:39.858598 16208 sys_catalog.cc:564] T 00000000000000000000000000000000 P cbfc33b15af8455191f1da2e10d7901f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:39.859736 16217 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbfc33b15af8455191f1da2e10d7901f [sys.catalog]: SysCatalogTable state changed. Reason: New leader cbfc33b15af8455191f1da2e10d7901f. Latest consensus state: current_term: 1 leader_uuid: "cbfc33b15af8455191f1da2e10d7901f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbfc33b15af8455191f1da2e10d7901f" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45231 } } }
I20260811 05:13:39.859880 16217 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbfc33b15af8455191f1da2e10d7901f [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:39.860164 16216 sys_catalog.cc:455] T 00000000000000000000000000000000 P cbfc33b15af8455191f1da2e10d7901f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cbfc33b15af8455191f1da2e10d7901f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cbfc33b15af8455191f1da2e10d7901f" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45231 } } }
I20260811 05:13:39.860285 16228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:39.860321 16216 sys_catalog.cc:458] T 00000000000000000000000000000000 P cbfc33b15af8455191f1da2e10d7901f [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:39.863842 16228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:39.870018 16228 catalog_manager.cc:1347] Generated new cluster ID: 1e427dd447a24b6d8ae74a64f95c0ba9
I20260811 05:13:39.870100 16228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:39.888576 16228 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:39.889662 16228 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:39.902547 16228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cbfc33b15af8455191f1da2e10d7901f: Generated new TSK 0
I20260811 05:13:39.903419 16228 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:13:39.927186 16211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:39.921994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49148 (local address 127.7.67.190:45231)
0811 05:13:39.922152 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:39.922157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:39.922350 (+ 193us) server_negotiation.cc:407] Connection header received
0811 05:13:39.922499 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:39.922501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:39.922565 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:39.922653 (+ 88us) server_negotiation.cc:226] Negotiated authn=JWT
0811 05:13:39.922950 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.923454 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.924786 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.925004 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.925086 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:39.925422 (+ 336us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 05:13:39.926820 (+ 1398us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34395/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 05:13:39.927047 (+ 227us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49148: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34395/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 05:13:39.927265 16211 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49148: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34395/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 05:13:39.929111 16253 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:39.921917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45231 (local address 127.0.0.1:49148)
0811 05:13:39.922252 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:39.922269 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:39.922389 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:39.922681 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:39.922684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:39.922692 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 05:13:39.922837 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:39.922844 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.923581 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.923585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:39.924663 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:39.924675 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:39.925130 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:39.925135 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:39.925225 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:39.925254 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 05:13:39.926994 (+ 1740us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34395/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 05:13:39.928908 (+ 1914us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.67.190:45231: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34395/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":1716,"thread_start_us":125,"threads_started":1}
I20260811 05:13:39.929659 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 16104
E20260811 05:13:39.950439 16255 webserver.cc:533] Webserver: sslize failed:
E20260811 05:13:39.950544 16255 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T05:13:39Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (359 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T05:13:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:39Z Disabled control of system clock
I20260811 05:13:39.997882 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:35513
--webserver_interface=127.7.67.190
--webserver_port=0
--builtin_ntp_servers=127.7.67.148:35041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:35513
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425190664461-7438-0/testchainca.cert with env {}
W20260811 05:13:40.114841 16260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:40.115097 16260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:40.115137 16260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:40.119251 16260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:40.119313 16260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:40.119328 16260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:40.119345 16260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:40.119360 16260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 05:13:40.119418 16260 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 05:13:40.119448 16260 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 05:13:40.125221 16260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:35041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:35513
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:35513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425190664461-7438-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:40.126286 16260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:40.127310 16260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:40.135891 16270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:40.135866 16277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:40.135917 16272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:40.136268 16260 server_base.cc:1034] running on GCE node
I20260811 05:13:40.136929 16260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:40.137984 16260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:40.139173 16260 hybrid_clock.cc:648] HybridClock initialized: now 1786425220139158 us; error 27 us; skew 500 ppm
I20260811 05:13:40.141124 16260 webserver.cc:460] Webserver started at http://127.7.67.190:44815/ using document root <none> and password file <none>
I20260811 05:13:40.141752 16260 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:40.141849 16260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:40.142089 16260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:40.144181 16260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "c9c7332801214d55be34de9e2cbbdbfa"
format_stamp: "Formatted at 2026-08-11 05:13:40 on dist-test-slave-04bb"
server_key: "28c807e48bb6cb5f3cde8c7a734ac170"
server_key_iv: "a17fa26e651b37312ed9f2cb1a899d0e"
server_key_version: "encryptionkey@0"
I20260811 05:13:40.144804 16260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "c9c7332801214d55be34de9e2cbbdbfa"
format_stamp: "Formatted at 2026-08-11 05:13:40 on dist-test-slave-04bb"
server_key: "28c807e48bb6cb5f3cde8c7a734ac170"
server_key_iv: "a17fa26e651b37312ed9f2cb1a899d0e"
server_key_version: "encryptionkey@0"
I20260811 05:13:40.148461 16260 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 05:13:40.150970 16289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:40.152257 16260 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 05:13:40.152431 16260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "c9c7332801214d55be34de9e2cbbdbfa"
format_stamp: "Formatted at 2026-08-11 05:13:40 on dist-test-slave-04bb"
server_key: "28c807e48bb6cb5f3cde8c7a734ac170"
server_key_iv: "a17fa26e651b37312ed9f2cb1a899d0e"
server_key_version: "encryptionkey@0"
I20260811 05:13:40.152586 16260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:40.169075 16260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:40.169781 16260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:40.169991 16260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:40.177932 16260 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:35513
I20260811 05:13:40.177963 16376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:35513 every 8 connection(s)
I20260811 05:13:40.178972 16260 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:40.182233 16377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:40.183965 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 16260
I20260811 05:13:40.184101 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:13:40.184389 7438 external_mini_cluster.cc:1428] Setting key 02e22dcea19ce17516f4a6505960eb5a
I20260811 05:13:40.189218 16377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9c7332801214d55be34de9e2cbbdbfa: Bootstrap starting.
I20260811 05:13:40.191952 16377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9c7332801214d55be34de9e2cbbdbfa: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:40.192924 16377 log.cc:824] T 00000000000000000000000000000000 P c9c7332801214d55be34de9e2cbbdbfa: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:40.194633 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:40.185891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33852 (local address 127.7.67.190:35513)
0811 05:13:40.186662 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:40.186684 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:40.186726 (+ 42us) server_negotiation.cc:407] Connection header received
0811 05:13:40.187435 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:40.187456 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:40.187999 (+ 543us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:40.188330 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:40.188774 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:40.189835 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:40.191157 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:40.191637 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:40.191780 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:40.192785 (+ 1005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:40.192790 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:40.192803 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:13:40.193258 (+ 455us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:40.193318 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:13:40.193340 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:40.193424 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:40.193579 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 05:13:40.193869 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":143,"threads_started":1}
I20260811 05:13:40.195019 16377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9c7332801214d55be34de9e2cbbdbfa: No bootstrap required, opened a new log
I20260811 05:13:40.197846 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:13:40.199512 16377 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9c7332801214d55be34de9e2cbbdbfa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c7332801214d55be34de9e2cbbdbfa" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 35513 } }
I20260811 05:13:40.199792 16377 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9c7332801214d55be34de9e2cbbdbfa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:40.199918 16377 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9c7332801214d55be34de9e2cbbdbfa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9c7332801214d55be34de9e2cbbdbfa, State: Initialized, Role: FOLLOWER
I20260811 05:13:40.200377 16377 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9c7332801214d55be34de9e2cbbdbfa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c7332801214d55be34de9e2cbbdbfa" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 35513 } }
I20260811 05:13:40.200500 16377 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9c7332801214d55be34de9e2cbbdbfa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:40.200613 16377 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9c7332801214d55be34de9e2cbbdbfa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:40.200734 16377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9c7332801214d55be34de9e2cbbdbfa [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:40.201666 16377 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9c7332801214d55be34de9e2cbbdbfa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c7332801214d55be34de9e2cbbdbfa" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 35513 } }
I20260811 05:13:40.202010 16377 leader_election.cc:302] T 00000000000000000000000000000000 P c9c7332801214d55be34de9e2cbbdbfa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9c7332801214d55be34de9e2cbbdbfa; no voters:
I20260811 05:13:40.202358 16377 leader_election.cc:288] T 00000000000000000000000000000000 P c9c7332801214d55be34de9e2cbbdbfa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:40.202489 16384 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9c7332801214d55be34de9e2cbbdbfa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:40.202775 16384 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9c7332801214d55be34de9e2cbbdbfa [term 1 LEADER]: Becoming Leader. State: Replica: c9c7332801214d55be34de9e2cbbdbfa, State: Running, Role: LEADER
I20260811 05:13:40.203155 16384 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9c7332801214d55be34de9e2cbbdbfa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c7332801214d55be34de9e2cbbdbfa" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 35513 } }
I20260811 05:13:40.203600 16377 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9c7332801214d55be34de9e2cbbdbfa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:40.204859 16385 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9c7332801214d55be34de9e2cbbdbfa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9c7332801214d55be34de9e2cbbdbfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c7332801214d55be34de9e2cbbdbfa" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 35513 } } }
I20260811 05:13:40.204993 16385 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9c7332801214d55be34de9e2cbbdbfa [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:40.205407 16397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:40.206283 16386 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9c7332801214d55be34de9e2cbbdbfa [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9c7332801214d55be34de9e2cbbdbfa. Latest consensus state: current_term: 1 leader_uuid: "c9c7332801214d55be34de9e2cbbdbfa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9c7332801214d55be34de9e2cbbdbfa" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 35513 } } }
I20260811 05:13:40.206384 16386 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9c7332801214d55be34de9e2cbbdbfa [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:40.208796 16397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:40.213951 16397 catalog_manager.cc:1347] Generated new cluster ID: 21d717a6f0d14f0889ed5b21211d3568
I20260811 05:13:40.214017 16397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:40.240371 16397 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:40.241364 16397 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:40.245428 16397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9c7332801214d55be34de9e2cbbdbfa: Generated new TSK 0
I20260811 05:13:40.246169 16397 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:13:40.734464 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:40.255706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33864 (local address 127.7.67.190:35513)
0811 05:13:40.255858 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:40.255863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:40.256017 (+ 154us) server_negotiation.cc:407] Connection header received
0811 05:13:40.256147 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:40.256150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:40.256206 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:40.256319 (+ 113us) server_negotiation.cc:226] Negotiated authn=JWT
0811 05:13:40.256625 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:40.257490 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:40.258418 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:40.258782 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:40.258884 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:40.259174 (+ 290us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 05:13:40.734049 (+474875us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 05:13:40.734338 (+ 289us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33864: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 05:13:40.734576 16382 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33864: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 05:13:40.734706 16425 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:40.255575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:35513 (local address 127.0.0.1:33864)
0811 05:13:40.255930 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:40.255944 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:40.256039 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:40.256359 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:40.256364 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:40.256376 (+ 12us) client_negotiation.cc:189] Negotiated authn=JWT
0811 05:13:40.256489 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:40.256494 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:40.257602 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:40.257605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:40.258271 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:40.258280 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:40.258897 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:40.258899 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:40.258987 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:40.259009 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 05:13:40.734306 (+475297us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 05:13:40.734500 (+ 194us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.7.67.190:35513: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":140,"threads_started":1}
I20260811 05:13:40.735431 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 16260
2026-08-11T05:13:40Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (775 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (131 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (132 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:13:41 dist-test-slave-04bb krb5kdc[16442](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:13:41 dist-test-slave-04bb krb5kdc[16442](info): set up 2 sockets
Aug 11 05:13:41 dist-test-slave-04bb krb5kdc[16442](info): commencing operation
krb5kdc: starting...
W20260811 05:13:42.265950 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:13:42 dist-test-slave-04bb krb5kdc[16442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:13:42.396122 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:36731
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:41621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:36731
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:42.517473 16465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:42.517721 16465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:42.517761 16465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:42.521761 16465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:42.521817 16465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:42.521836 16465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:42.521853 16465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:42.521868 16465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:42.527159 16465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:41621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:36731
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:36731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:42.528193 16465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:42.529245 16465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:42.538305 16477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:42.538424 16481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:42.538393 16465 server_base.cc:1034] running on GCE node
W20260811 05:13:42.538301 16473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:42.539273 16465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:42.540402 16465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:42.541632 16465 hybrid_clock.cc:648] HybridClock initialized: now 1786425222541592 us; error 34 us; skew 500 ppm
Aug 11 05:13:42 dist-test-slave-04bb krb5kdc[16442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425222, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:42.547880 16465 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:13:42.548878 16465 webserver.cc:460] Webserver started at http://127.7.67.190:38641/ using document root <none> and password file <none>
I20260811 05:13:42.549453 16465 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:42.549520 16465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:42.549706 16465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:42.551852 16465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "c3fa88fd2e6a4adb9e8774d8939da4be"
format_stamp: "Formatted at 2026-08-11 05:13:42 on dist-test-slave-04bb"
server_key: "11dffa3578c19d09f84fff04e9908717"
server_key_iv: "c0754494f5fd8b7ecd50b63fb969f18e"
server_key_version: "encryptionkey@0"
I20260811 05:13:42.552467 16465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "c3fa88fd2e6a4adb9e8774d8939da4be"
format_stamp: "Formatted at 2026-08-11 05:13:42 on dist-test-slave-04bb"
server_key: "11dffa3578c19d09f84fff04e9908717"
server_key_iv: "c0754494f5fd8b7ecd50b63fb969f18e"
server_key_version: "encryptionkey@0"
I20260811 05:13:42.556443 16465 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 05:13:42.559095 16491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:42.560426 16465 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:13:42.560607 16465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "c3fa88fd2e6a4adb9e8774d8939da4be"
format_stamp: "Formatted at 2026-08-11 05:13:42 on dist-test-slave-04bb"
server_key: "11dffa3578c19d09f84fff04e9908717"
server_key_iv: "c0754494f5fd8b7ecd50b63fb969f18e"
server_key_version: "encryptionkey@0"
I20260811 05:13:42.560736 16465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:42.593748 16465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:42.597728 16465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:42.597990 16465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:42.606415 16465 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:36731
I20260811 05:13:42.606424 16581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:36731 every 8 connection(s)
I20260811 05:13:42.607527 16465 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:42.610458 16584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:42.613413 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 16465
I20260811 05:13:42.613545 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:13:42.613866 7438 external_mini_cluster.cc:1428] Setting key 3bf5d01f52ebb723d265d52ec3baad3d
I20260811 05:13:42.616824 16584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3fa88fd2e6a4adb9e8774d8939da4be: Bootstrap starting.
I20260811 05:13:42.619406 16584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3fa88fd2e6a4adb9e8774d8939da4be: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:42.620301 16584 log.cc:824] T 00000000000000000000000000000000 P c3fa88fd2e6a4adb9e8774d8939da4be: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:42.622859 16584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3fa88fd2e6a4adb9e8774d8939da4be: No bootstrap required, opened a new log
Aug 11 05:13:42 dist-test-slave-04bb krb5kdc[16442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425222, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:42.626191 16584 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3fa88fd2e6a4adb9e8774d8939da4be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3fa88fd2e6a4adb9e8774d8939da4be" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 36731 } }
I20260811 05:13:42.626442 16584 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3fa88fd2e6a4adb9e8774d8939da4be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:42.626525 16584 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3fa88fd2e6a4adb9e8774d8939da4be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3fa88fd2e6a4adb9e8774d8939da4be, State: Initialized, Role: FOLLOWER
I20260811 05:13:42.627097 16584 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3fa88fd2e6a4adb9e8774d8939da4be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3fa88fd2e6a4adb9e8774d8939da4be" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 36731 } }
I20260811 05:13:42.627275 16584 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3fa88fd2e6a4adb9e8774d8939da4be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:42.627344 16584 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3fa88fd2e6a4adb9e8774d8939da4be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:42.627450 16584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3fa88fd2e6a4adb9e8774d8939da4be [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:42.628547 16584 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3fa88fd2e6a4adb9e8774d8939da4be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3fa88fd2e6a4adb9e8774d8939da4be" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 36731 } }
I20260811 05:13:42.628974 16584 leader_election.cc:302] T 00000000000000000000000000000000 P c3fa88fd2e6a4adb9e8774d8939da4be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3fa88fd2e6a4adb9e8774d8939da4be; no voters:
I20260811 05:13:42.629323 16584 leader_election.cc:288] T 00000000000000000000000000000000 P c3fa88fd2e6a4adb9e8774d8939da4be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:42.629475 16592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3fa88fd2e6a4adb9e8774d8939da4be [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:42.629730 16592 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3fa88fd2e6a4adb9e8774d8939da4be [term 1 LEADER]: Becoming Leader. State: Replica: c3fa88fd2e6a4adb9e8774d8939da4be, State: Running, Role: LEADER
I20260811 05:13:42.630065 16592 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3fa88fd2e6a4adb9e8774d8939da4be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3fa88fd2e6a4adb9e8774d8939da4be" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 36731 } }
I20260811 05:13:42.630702 16584 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3fa88fd2e6a4adb9e8774d8939da4be [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:42.631908 16593 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3fa88fd2e6a4adb9e8774d8939da4be [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3fa88fd2e6a4adb9e8774d8939da4be. Latest consensus state: current_term: 1 leader_uuid: "c3fa88fd2e6a4adb9e8774d8939da4be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3fa88fd2e6a4adb9e8774d8939da4be" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 36731 } } }
I20260811 05:13:42.631938 16594 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3fa88fd2e6a4adb9e8774d8939da4be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3fa88fd2e6a4adb9e8774d8939da4be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3fa88fd2e6a4adb9e8774d8939da4be" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 36731 } } }
I20260811 05:13:42.632046 16593 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3fa88fd2e6a4adb9e8774d8939da4be [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:42.632074 16594 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3fa88fd2e6a4adb9e8774d8939da4be [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:42.632500 16597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:42.632687 16588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:42.615417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37602 (local address 127.7.67.190:36731)
0811 05:13:42.615922 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:42.615944 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:42.616011 (+ 67us) server_negotiation.cc:407] Connection header received
0811 05:13:42.616777 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:42.616799 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:42.617224 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:42.617534 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:42.618494 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:42.619702 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:42.620877 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:42.621565 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:42.621724 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:42.624872 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:42.624879 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:42.624894 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:42.624924 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:42.627737 (+ 2813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:42.628268 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:42.628275 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:42.628281 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:42.628445 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:42.628839 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:42.628842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:42.628844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:42.629418 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:42.629635 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:42.630634 (+ 999us) server_negotiation.cc:299] Negotiation successful
0811 05:13:42.630957 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":183,"threads_started":1}
I20260811 05:13:42.635651 16597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:42.641711 16597 catalog_manager.cc:1347] Generated new cluster ID: 387034e632a8434292caaa1c6a254472
I20260811 05:13:42.641786 16597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:42.648382 16597 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:42.649703 16597 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:42.655730 16597 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3fa88fd2e6a4adb9e8774d8939da4be: Generated new TSK 0
I20260811 05:13:42.656374 16597 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
I20260811 05:13:42.691713 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:36731
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:41621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:42.811820 16619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:42.812084 16619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:42.812139 16619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:42.816156 16619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:42.816218 16619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:42.816277 16619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:13:42.821449 16619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:41621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:36731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:42.822424 16619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:42.823406 16619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:42.833913 16625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:42.833909 16627 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:42.834260 16619 server_base.cc:1034] running on GCE node
W20260811 05:13:42.834321 16630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:42.834724 16619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:42.835386 16619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:42.836629 16619 hybrid_clock.cc:648] HybridClock initialized: now 1786425222836593 us; error 44 us; skew 500 ppm
Aug 11 05:13:42 dist-test-slave-04bb krb5kdc[16442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425222, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:42.840005 16619 init.cc:377] Logged in from keytab as kudu/127.7.67.129@KRBTEST.COM (short username kudu)
I20260811 05:13:42.840914 16619 webserver.cc:460] Webserver started at http://127.7.67.129:46631/ using document root <none> and password file <none>
I20260811 05:13:42.841471 16619 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:42.841532 16619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:42.841748 16619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:42.843726 16619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "43e99ae85a40472aaec7dae80d998ead"
format_stamp: "Formatted at 2026-08-11 05:13:42 on dist-test-slave-04bb"
server_key: "ed74caf2b9f10e0b9cb18806a6274e2a"
server_key_iv: "15f83be17a0a93b9644b019ef14190cb"
server_key_version: "encryptionkey@0"
I20260811 05:13:42.844291 16619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "43e99ae85a40472aaec7dae80d998ead"
format_stamp: "Formatted at 2026-08-11 05:13:42 on dist-test-slave-04bb"
server_key: "ed74caf2b9f10e0b9cb18806a6274e2a"
server_key_iv: "15f83be17a0a93b9644b019ef14190cb"
server_key_version: "encryptionkey@0"
I20260811 05:13:42.847954 16619 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 05:13:42.850610 16637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:42.851883 16619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 05:13:42.851984 16619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "43e99ae85a40472aaec7dae80d998ead"
format_stamp: "Formatted at 2026-08-11 05:13:42 on dist-test-slave-04bb"
server_key: "ed74caf2b9f10e0b9cb18806a6274e2a"
server_key_iv: "15f83be17a0a93b9644b019ef14190cb"
server_key_version: "encryptionkey@0"
I20260811 05:13:42.852113 16619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:42.867050 16619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:42.870229 16619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:42.870466 16619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:42.871102 16619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:42.871948 16619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:42.871994 16619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:42.872064 16619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:42.872097 16619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:42.882150 16619 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:36031
I20260811 05:13:42.882171 16837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:36031 every 8 connection(s)
I20260811 05:13:42.883095 16619 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:42.889214 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 16619
I20260811 05:13:42.889343 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:13:42.889689 7438 external_mini_cluster.cc:1428] Setting key c75ee0d893db2421b69ba22c8c0d6400
Aug 11 05:13:42 dist-test-slave-04bb krb5kdc[16442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425222, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:42.898780 16588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:42.885233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:50041 (local address 127.7.67.190:36731)
0811 05:13:42.885397 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:42.885403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:42.886360 (+ 957us) server_negotiation.cc:407] Connection header received
0811 05:13:42.887946 (+ 1586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:42.887950 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:42.888002 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:42.888089 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:42.889478 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:42.890106 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:42.891265 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:42.891503 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:42.891612 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:42.894483 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:42.894487 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:42.894489 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:42.894515 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:42.896074 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:42.896891 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:42.896897 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:42.896898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:42.896992 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:42.897482 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:42.897485 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:42.897486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:42.897664 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:42.897742 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:42.898542 (+ 800us) server_negotiation.cc:299] Negotiation successful
0811 05:13:42.898654 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 05:13:42.900173 16840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:42.885687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:36731 (local address 127.7.67.129:50041)
0811 05:13:42.886190 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:42.886256 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:42.887695 (+ 1439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:42.888292 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:42.888299 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:42.888749 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:42.889259 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:42.889273 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:42.890244 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:42.890248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:42.891118 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:42.891127 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:42.891667 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:42.891671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:42.891840 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:42.892486 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:42.892503 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:42.894268 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:42.896215 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:42.896226 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:42.896247 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:42.896745 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:42.897110 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:42.897126 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:42.897129 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:42.897358 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:42.897788 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:42.897800 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:42.898210 (+ 410us) client_negotiation.cc:769] Sending connection context
0811 05:13:42.898481 (+ 271us) client_negotiation.cc:240] Negotiation successful
0811 05:13:42.898911 (+ 430us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":148,"threads_started":1}
I20260811 05:13:42.901587 16838 heartbeater.cc:347] Connected to a master server at 127.7.67.190:36731
I20260811 05:13:42.901852 16838 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.7.67.130@KRBTEST.COM; defaulting to no policy
I20260811 05:13:42.902375 16838 heartbeater.cc:511] Master 127.7.67.190:36731 requested a full tablet report, sending...
I20260811 05:13:42.903967 16517 ts_manager.cc:194] Registered new tserver with Master: 43e99ae85a40472aaec7dae80d998ead (127.7.67.129:36031)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.130@KRBTEST.COM" created.
I20260811 05:13:42.905398 16517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.129@KRBTEST.COM'} at 127.7.67.129:50041
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
I20260811 05:13:42.923326 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:36731
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:41621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:43.035488 16844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:43.035722 16844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:43.035763 16844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:43.039687 16844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:43.039736 16844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:43.039824 16844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:13:43.044934 16844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:41621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:36731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:43.045897 16844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:43.046911 16844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:43.056422 16851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:43.056547 16856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:43.056650 16852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:43.056883 16844 server_base.cc:1034] running on GCE node
I20260811 05:13:43.057300 16844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:43.057986 16844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:43.059163 16844 hybrid_clock.cc:648] HybridClock initialized: now 1786425223059139 us; error 33 us; skew 500 ppm
Aug 11 05:13:43 dist-test-slave-04bb krb5kdc[16442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425223, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:43.062227 16844 init.cc:377] Logged in from keytab as kudu/127.7.67.130@KRBTEST.COM (short username kudu)
I20260811 05:13:43.063166 16844 webserver.cc:460] Webserver started at http://127.7.67.130:32773/ using document root <none> and password file <none>
I20260811 05:13:43.063781 16844 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:43.063879 16844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:43.064170 16844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:43.066752 16844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "9592bee70cb44409827f17c2745ece89"
format_stamp: "Formatted at 2026-08-11 05:13:43 on dist-test-slave-04bb"
server_key: "d05cf8f079a7db1b5d06b8bdb2a4c0d7"
server_key_iv: "4c8f2d6cfb732b92ffc25c80cb06a3a0"
server_key_version: "encryptionkey@0"
I20260811 05:13:43.067390 16844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "9592bee70cb44409827f17c2745ece89"
format_stamp: "Formatted at 2026-08-11 05:13:43 on dist-test-slave-04bb"
server_key: "d05cf8f079a7db1b5d06b8bdb2a4c0d7"
server_key_iv: "4c8f2d6cfb732b92ffc25c80cb06a3a0"
server_key_version: "encryptionkey@0"
I20260811 05:13:43.071769 16844 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 05:13:43.074559 16871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:43.075893 16844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 05:13:43.076046 16844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "9592bee70cb44409827f17c2745ece89"
format_stamp: "Formatted at 2026-08-11 05:13:43 on dist-test-slave-04bb"
server_key: "d05cf8f079a7db1b5d06b8bdb2a4c0d7"
server_key_iv: "4c8f2d6cfb732b92ffc25c80cb06a3a0"
server_key_version: "encryptionkey@0"
I20260811 05:13:43.076174 16844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:43.100962 16844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:43.104729 16844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:43.104954 16844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:43.105633 16844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:43.106539 16844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:43.106616 16844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:43.106693 16844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:43.106743 16844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:43.116605 16844 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:40547
I20260811 05:13:43.116634 17052 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:40547 every 8 connection(s)
I20260811 05:13:43.117556 16844 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:43.120299 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 16844
I20260811 05:13:43.120442 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:13:43.120805 7438 external_mini_cluster.cc:1428] Setting key fa76d2da538df131772c9297988eeafd
Aug 11 05:13:43 dist-test-slave-04bb krb5kdc[16442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425223, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.131@KRBTEST.COM; defaulting to no policy
I20260811 05:13:43.134153 16588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.119534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:56911 (local address 127.7.67.190:36731)
0811 05:13:43.119693 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:43.119697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:43.120725 (+ 1028us) server_negotiation.cc:407] Connection header received
0811 05:13:43.122333 (+ 1608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:43.122337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:43.122394 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:43.122491 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:43.124119 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.124841 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.126554 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.127042 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.127171 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:43.129687 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:43.129696 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:43.129701 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:43.129742 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:43.131688 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:43.132392 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:43.132398 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:43.132400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:43.132511 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:43.132894 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:43.132901 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:43.132906 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:43.133195 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:43.133365 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:43.133912 (+ 547us) server_negotiation.cc:299] Negotiation successful
0811 05:13:43.134023 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 05:13:43.135126 17059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.119914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:36731 (local address 127.7.67.130:56911)
0811 05:13:43.120562 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:43.120627 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:43.122075 (+ 1448us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:43.122692 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:43.122702 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:43.123228 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:43.123950 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:43.123973 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.124982 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.124988 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:43.126420 (+ 1432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:43.126430 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.127192 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.127195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:43.127362 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:43.127995 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:43.128011 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:43.129497 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:43.131824 (+ 2327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:43.131829 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:43.131841 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:43.132274 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:43.132637 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:43.132640 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:43.132641 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:43.132780 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:43.133360 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:43.133367 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:43.133656 (+ 289us) client_negotiation.cc:769] Sending connection context
0811 05:13:43.133832 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 05:13:43.134156 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":166,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.131@KRBTEST.COM" created.
I20260811 05:13:43.136597 17055 heartbeater.cc:347] Connected to a master server at 127.7.67.190:36731
I20260811 05:13:43.136857 17055 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:43.137382 17055 heartbeater.cc:511] Master 127.7.67.190:36731 requested a full tablet report, sending...
I20260811 05:13:43.138509 16517 ts_manager.cc:194] Registered new tserver with Master: 9592bee70cb44409827f17c2745ece89 (127.7.67.130:40547)
I20260811 05:13:43.139104 16517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.130@KRBTEST.COM'} at 127.7.67.130:56911
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
I20260811 05:13:43.153586 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:36731
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:41621
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:43.281049 17062 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:43.281301 17062 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:43.281347 17062 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:43.285557 17062 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:43.285606 17062 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:43.285663 17062 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:13:43.290771 17062 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:41621
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:36731
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17062
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:43.291682 17062 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:43.292668 17062 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:43.302129 17073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:43.302238 17071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:43.302129 17070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:43.302553 17062 server_base.cc:1034] running on GCE node
I20260811 05:13:43.302948 17062 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:43.303591 17062 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:43.304746 17062 hybrid_clock.cc:648] HybridClock initialized: now 1786425223304731 us; error 25 us; skew 500 ppm
Aug 11 05:13:43 dist-test-slave-04bb krb5kdc[16442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425223, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:43.307412 17062 init.cc:377] Logged in from keytab as kudu/127.7.67.131@KRBTEST.COM (short username kudu)
I20260811 05:13:43.308282 17062 webserver.cc:460] Webserver started at http://127.7.67.131:46641/ using document root <none> and password file <none>
I20260811 05:13:43.308887 17062 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:43.308949 17062 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:43.309175 17062 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:43.311280 17062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "fb025b16f49d4e9f91991870b14b4c26"
format_stamp: "Formatted at 2026-08-11 05:13:43 on dist-test-slave-04bb"
server_key: "a3415b47ce60a810cd5f9a8593b0a834"
server_key_iv: "74bb5361eeffcbbb674d5a059ec386eb"
server_key_version: "encryptionkey@0"
I20260811 05:13:43.311821 17062 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "fb025b16f49d4e9f91991870b14b4c26"
format_stamp: "Formatted at 2026-08-11 05:13:43 on dist-test-slave-04bb"
server_key: "a3415b47ce60a810cd5f9a8593b0a834"
server_key_iv: "74bb5361eeffcbbb674d5a059ec386eb"
server_key_version: "encryptionkey@0"
I20260811 05:13:43.315410 17062 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 05:13:43.317857 17083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:43.319064 17062 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:43.319232 17062 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "fb025b16f49d4e9f91991870b14b4c26"
format_stamp: "Formatted at 2026-08-11 05:13:43 on dist-test-slave-04bb"
server_key: "a3415b47ce60a810cd5f9a8593b0a834"
server_key_iv: "74bb5361eeffcbbb674d5a059ec386eb"
server_key_version: "encryptionkey@0"
I20260811 05:13:43.319343 17062 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:43.339007 17062 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:43.341892 17062 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:43.342085 17062 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:43.342649 17062 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:43.343487 17062 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:43.343554 17062 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:43.343617 17062 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:43.343662 17062 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:43.352921 17062 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:41723
I20260811 05:13:43.352949 17255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:41723 every 8 connection(s)
I20260811 05:13:43.353771 17062 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:13:43.361370 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 17062
I20260811 05:13:43.361464 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:13:43.361756 7438 external_mini_cluster.cc:1428] Setting key 896b716de44a823ae775b0afb99a821e
Aug 11 05:13:43 dist-test-slave-04bb krb5kdc[16442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425223, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:43.367560 16588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.355518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:48425 (local address 127.7.67.190:36731)
0811 05:13:43.355650 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:43.355653 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:43.356477 (+ 824us) server_negotiation.cc:407] Connection header received
0811 05:13:43.357924 (+ 1447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:43.357927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:43.357976 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:43.358052 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:43.359173 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.359695 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.360759 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.360968 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.361048 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:43.363926 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:43.363935 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:43.363939 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:43.363975 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:43.365633 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:43.366164 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:43.366167 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:43.366168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:43.366263 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:43.366572 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:43.366575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:43.366576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:43.366776 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:43.366868 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:43.367312 (+ 444us) server_negotiation.cc:299] Negotiation successful
0811 05:13:43.367429 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 05:13:43.368335 17258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.355874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:36731 (local address 127.7.67.131:48425)
0811 05:13:43.356326 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:43.356366 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:43.357709 (+ 1343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:43.358216 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:43.358222 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:43.358551 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:43.359028 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:43.359041 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.359808 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.359811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:43.360654 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:43.360663 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.361112 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.361115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:43.361265 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:43.361950 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:43.361967 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:43.363668 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:43.365768 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:43.365773 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:43.365784 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:43.366052 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:43.366352 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:43.366354 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:43.366355 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:43.366483 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:43.366876 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:43.366881 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:43.367093 (+ 212us) client_negotiation.cc:769] Sending connection context
0811 05:13:43.367243 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 05:13:43.367484 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":113,"threads_started":1}
I20260811 05:13:43.369504 17256 heartbeater.cc:347] Connected to a master server at 127.7.67.190:36731
I20260811 05:13:43.369745 17256 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:43.370218 17256 heartbeater.cc:511] Master 127.7.67.190:36731 requested a full tablet report, sending...
I20260811 05:13:43.371230 16517 ts_manager.cc:194] Registered new tserver with Master: fb025b16f49d4e9f91991870b14b4c26 (127.7.67.131:41723)
I20260811 05:13:43.371778 16517 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.131@KRBTEST.COM'} at 127.7.67.131:48425
I20260811 05:13:43.375728 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/test-admin.keytab.
Aug 11 05:13:43 dist-test-slave-04bb krb5kdc[16442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:43.393397 7438 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 05:13:43.396420 17269 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:43.394890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57990 (local address 127.7.67.129:36031)
0811 05:13:43.395242 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:43.395262 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:43.395285 (+ 23us) server_negotiation.cc:407] Connection header received
0811 05:13:43.395337 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:43.395342 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:43.395525 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:43.395680 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:43.396241 (+ 561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57990: BlockingRecv error: recv got EOF from 127.0.0.1:57990 (error 108)
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":93,"threads_started":1}
W20260811 05:13:43.396766 7438 init.cc:244] Error while opening credential cache '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/krb5cc)
Aug 11 05:13:43 dist-test-slave-04bb krb5kdc[16442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:43.397759 7438 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 05:13:43 dist-test-slave-04bb krb5kdc[16442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:43.398665 7438 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 05:13:43 dist-test-slave-04bb krb5kdc[16442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.129@KRBTEST.COM
I20260811 05:13:43.408164 17269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.399492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57992 (local address 127.7.67.129:36031)
0811 05:13:43.399594 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:43.399598 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:43.399751 (+ 153us) server_negotiation.cc:407] Connection header received
0811 05:13:43.399853 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:43.399856 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:43.399896 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:43.399963 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:43.400471 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.401332 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.402160 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.402542 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.402628 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:43.404952 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:43.404961 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:43.404975 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:43.404997 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:43.406571 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:43.406957 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:43.406962 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:43.406963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:43.407039 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:43.407268 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:43.407271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:43.407272 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:43.407501 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:43.407646 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:43.407841 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 05:13:43.408032 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260811 05:13:43.412245 17269 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:43.411153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58006 (local address 127.7.67.129:36031)
0811 05:13:43.411296 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:43.411301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:43.411457 (+ 156us) server_negotiation.cc:407] Connection header received
0811 05:13:43.411605 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:43.411613 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:43.411649 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:43.411744 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:43.412132 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58006: BlockingRecv error: recv got EOF from 127.0.0.1:58006 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 05:13:43.412412 7438 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/krb5cc' of type FILE
Aug 11 05:13:43 dist-test-slave-04bb krb5kdc[16442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:43.413472 7438 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 05:13:43 dist-test-slave-04bb krb5kdc[16442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:43.414333 7438 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 05:13:43 dist-test-slave-04bb krb5kdc[16442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.129@KRBTEST.COM
I20260811 05:13:43.422816 17269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.415093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58016 (local address 127.7.67.129:36031)
0811 05:13:43.415225 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:43.415228 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:43.415242 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:13:43.415345 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:43.415347 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:43.415387 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:43.415454 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:43.415977 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.416475 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.417305 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.417544 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.417622 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:43.419753 (+ 2131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:43.419756 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:43.419759 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:43.419780 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:43.421285 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:43.421649 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:43.421651 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:43.421652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:43.421724 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:43.421957 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:43.421960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:43.421961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:43.422107 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:43.422201 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:43.422453 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 05:13:43.422593 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 05:13:43.425652 17269 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:43.424655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58022 (local address 127.7.67.129:36031)
0811 05:13:43.424778 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:43.424782 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:43.424923 (+ 141us) server_negotiation.cc:407] Connection header received
0811 05:13:43.425040 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:43.425043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:43.425079 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:43.425179 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:43.425561 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58022: BlockingRecv error: recv got EOF from 127.0.0.1:58022 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 05:13:43.425796 7438 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786425190664461-7438-0/krb5kdc/krb5cc' of type FILE
Aug 11 05:13:43 dist-test-slave-04bb krb5kdc[16442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:43.426851 7438 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 05:13:43 dist-test-slave-04bb krb5kdc[16442](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:43.427915 7438 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 05:13:43 dist-test-slave-04bb krb5kdc[16442](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.129@KRBTEST.COM
I20260811 05:13:43.437260 17269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:43.428738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58026 (local address 127.7.67.129:36031)
0811 05:13:43.428855 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:43.428860 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:43.429036 (+ 176us) server_negotiation.cc:407] Connection header received
0811 05:13:43.429140 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:43.429142 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:43.429173 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:43.429237 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:43.429793 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.430270 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.431343 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:43.431575 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:43.431770 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:43.434151 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:43.434160 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:43.434162 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:43.434184 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:43.435575 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:43.436014 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:43.436018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:43.436019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:43.436092 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:43.436385 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:43.436388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:43.436390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:43.436608 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:43.436774 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:43.436996 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 05:13:43.437101 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 05:13:43.438259 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 16619
I20260811 05:13:43.451371 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 16844
I20260811 05:13:43.471998 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 17062
I20260811 05:13:43.495157 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 16465
2026-08-11T05:13:43Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2495 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:13:43 dist-test-slave-04bb krb5kdc[17292](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:13:43 dist-test-slave-04bb krb5kdc[17292](info): set up 2 sockets
Aug 11 05:13:43 dist-test-slave-04bb krb5kdc[17292](info): commencing operation
krb5kdc: starting...
W20260811 05:13:44.766391 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:13:44 dist-test-slave-04bb krb5kdc[17292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:44Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.190.keytab.
Entry for principal oryx/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.190.keytab.
I20260811 05:13:44.896656 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:45385
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.190.keytab
--principal=oryx/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:45919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:45385
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:45.016845 17321 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:45.017148 17321 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:45.017228 17321 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:45.021283 17321 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:45.021370 17321 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:45.021413 17321 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:45.021473 17321 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:45.021515 17321 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:45.026911 17321 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:45919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:45385
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.190.keytab
--principal=oryx/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:45385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17321
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:45.028002 17321 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:45.029072 17321 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:45.037662 17328 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:45.037694 17321 server_base.cc:1034] running on GCE node
W20260811 05:13:45.037642 17332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:45.037652 17327 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:45.038501 17321 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:45.039535 17321 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:45.040705 17321 hybrid_clock.cc:648] HybridClock initialized: now 1786425225040674 us; error 43 us; skew 500 ppm
Aug 11 05:13:45 dist-test-slave-04bb krb5kdc[17292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425225, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:45.043630 17321 init.cc:377] Logged in from keytab as oryx/127.7.67.190@KRBTEST.COM (short username oryx)
I20260811 05:13:45.044667 17321 webserver.cc:460] Webserver started at http://127.7.67.190:38259/ using document root <none> and password file <none>
I20260811 05:13:45.045187 17321 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:45.045253 17321 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:45.045423 17321 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:45.047401 17321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "95667e6974cb44ba9111427a619ddcef"
format_stamp: "Formatted at 2026-08-11 05:13:45 on dist-test-slave-04bb"
server_key: "cb398a53c7c516da77c1dd798100aaa1"
server_key_iv: "98ed2601e645fbe0550bf77613fbab92"
server_key_version: "encryptionkey@0"
I20260811 05:13:45.047948 17321 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "95667e6974cb44ba9111427a619ddcef"
format_stamp: "Formatted at 2026-08-11 05:13:45 on dist-test-slave-04bb"
server_key: "cb398a53c7c516da77c1dd798100aaa1"
server_key_iv: "98ed2601e645fbe0550bf77613fbab92"
server_key_version: "encryptionkey@0"
I20260811 05:13:45.051743 17321 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 05:13:45.054275 17340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:45.055469 17321 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:45.055583 17321 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "95667e6974cb44ba9111427a619ddcef"
format_stamp: "Formatted at 2026-08-11 05:13:45 on dist-test-slave-04bb"
server_key: "cb398a53c7c516da77c1dd798100aaa1"
server_key_iv: "98ed2601e645fbe0550bf77613fbab92"
server_key_version: "encryptionkey@0"
I20260811 05:13:45.055670 17321 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:45.083772 17321 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:45.087090 17321 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:45.087255 17321 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:45.095329 17321 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:45385
I20260811 05:13:45.095599 17422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:45385 every 8 connection(s)
I20260811 05:13:45.096378 17321 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:45.099150 17423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:45.104146 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 17321
I20260811 05:13:45.104283 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:13:45.104647 7438 external_mini_cluster.cc:1428] Setting key e113a079edef3cf05debf753ab2a808b
I20260811 05:13:45.104887 17423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef: Bootstrap starting.
I20260811 05:13:45.107365 17423 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:45.108310 17423 log.cc:824] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:45.110421 17423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef: No bootstrap required, opened a new log
Aug 11 05:13:45 dist-test-slave-04bb krb5kdc[17292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM
I20260811 05:13:45.114748 17423 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } }
I20260811 05:13:45.114977 17423 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:45.115069 17423 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95667e6974cb44ba9111427a619ddcef, State: Initialized, Role: FOLLOWER
I20260811 05:13:45.115537 17423 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } }
I20260811 05:13:45.115686 17423 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:45.115773 17423 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:45.115896 17423 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:45.116861 17423 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } }
I20260811 05:13:45.117270 17423 leader_election.cc:302] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95667e6974cb44ba9111427a619ddcef; no voters:
I20260811 05:13:45.117600 17423 leader_election.cc:288] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:45.117708 17431 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:45.117998 17431 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [term 1 LEADER]: Becoming Leader. State: Replica: 95667e6974cb44ba9111427a619ddcef, State: Running, Role: LEADER
I20260811 05:13:45.118364 17431 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } }
I20260811 05:13:45.119050 17423 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:45.120316 17437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95667e6974cb44ba9111427a619ddcef. Latest consensus state: current_term: 1 leader_uuid: "95667e6974cb44ba9111427a619ddcef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } }
I20260811 05:13:45.120425 17429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:45.105995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53806 (local address 127.7.67.190:45385)
0811 05:13:45.106485 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:45.106500 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:45.106536 (+ 36us) server_negotiation.cc:407] Connection header received
0811 05:13:45.107156 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:45.107174 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:45.107543 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:45.107836 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:45.108785 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:45.109906 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:45.111006 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:45.111464 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:45.111582 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:45.114434 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:45.114439 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:45.114452 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:45.114478 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:45.116977 (+ 2499us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:45.117548 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:45.117553 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:45.117557 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:45.117768 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:45.118102 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:45.118105 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:45.118107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:45.118585 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:45.118795 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:45.119093 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 05:13:45.119387 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":124,"threads_started":1}
I20260811 05:13:45.120323 17436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95667e6974cb44ba9111427a619ddcef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } }
I20260811 05:13:45.120467 17437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:45.120647 17436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:45.121009 17449 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:45.124330 17449 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:45.129989 17449 catalog_manager.cc:1347] Generated new cluster ID: e9f81071b52b4f468d783cb68b56dd85
I20260811 05:13:45.130053 17449 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:45.136005 17449 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:45.137033 17449 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:45.142148 17449 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef: Generated new TSK 0
I20260811 05:13:45.142838 17449 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.67.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.129.keytab.
Entry for principal oryx/127.7.67.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.129.keytab.
I20260811 05:13:45.176052 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:45385
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.129.keytab
--principal=oryx/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:45919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:45.300187 17467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:45.300495 17467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:45.300604 17467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:45.304636 17467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:45.304725 17467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:45.304816 17467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:13:45.310019 17467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:45919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.129.keytab
--principal=oryx/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:45385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:45.311034 17467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:45.312310 17467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:45.322453 17473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:45.322500 17478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:45.322454 17476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:45.322896 17467 server_base.cc:1034] running on GCE node
I20260811 05:13:45.323284 17467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:45.323890 17467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:45.325078 17467 hybrid_clock.cc:648] HybridClock initialized: now 1786425225325056 us; error 32 us; skew 500 ppm
Aug 11 05:13:45 dist-test-slave-04bb krb5kdc[17292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425225, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:45.327795 17467 init.cc:377] Logged in from keytab as oryx/127.7.67.129@KRBTEST.COM (short username oryx)
I20260811 05:13:45.328711 17467 webserver.cc:460] Webserver started at http://127.7.67.129:46679/ using document root <none> and password file <none>
I20260811 05:13:45.329221 17467 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:45.329276 17467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:45.329445 17467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:45.331348 17467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "d2e72a180c5e43a5ba0728584cc25be4"
format_stamp: "Formatted at 2026-08-11 05:13:45 on dist-test-slave-04bb"
server_key: "b89a5a680b302a9df49114421abd7463"
server_key_iv: "9f0cf358dbc9da2d70a9743d097976ba"
server_key_version: "encryptionkey@0"
I20260811 05:13:45.331866 17467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "d2e72a180c5e43a5ba0728584cc25be4"
format_stamp: "Formatted at 2026-08-11 05:13:45 on dist-test-slave-04bb"
server_key: "b89a5a680b302a9df49114421abd7463"
server_key_iv: "9f0cf358dbc9da2d70a9743d097976ba"
server_key_version: "encryptionkey@0"
I20260811 05:13:45.335532 17467 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 05:13:45.338001 17489 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:45.339241 17467 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:45.339356 17467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "d2e72a180c5e43a5ba0728584cc25be4"
format_stamp: "Formatted at 2026-08-11 05:13:45 on dist-test-slave-04bb"
server_key: "b89a5a680b302a9df49114421abd7463"
server_key_iv: "9f0cf358dbc9da2d70a9743d097976ba"
server_key_version: "encryptionkey@0"
I20260811 05:13:45.339457 17467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:45.362689 17467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:45.365679 17467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:45.365885 17467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:45.366461 17467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:45.367291 17467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:45.367363 17467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:45.367435 17467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:45.367479 17467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:45.376699 17467 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:36481
I20260811 05:13:45.376748 17670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:36481 every 8 connection(s)
I20260811 05:13:45.377611 17467 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:45.383213 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 17467
I20260811 05:13:45.383358 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:13:45.383642 7438 external_mini_cluster.cc:1428] Setting key 92b07042211a00b7debb3e6830975e49
Aug 11 05:13:45 dist-test-slave-04bb krb5kdc[17292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425225, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM
I20260811 05:13:45.393627 17429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:45.379409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:50159 (local address 127.7.67.190:45385)
0811 05:13:45.379554 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:45.379558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:45.380392 (+ 834us) server_negotiation.cc:407] Connection header received
0811 05:13:45.381719 (+ 1327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:45.381723 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:45.381782 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:45.381888 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:45.383053 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:45.383733 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:45.385340 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:45.385729 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:45.385893 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:45.389503 (+ 3610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:45.389506 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:45.389508 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:45.389534 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:45.391291 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:45.391882 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:45.391886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:45.391887 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:45.391973 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:45.392363 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:45.392367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:45.392369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:45.392649 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:45.392770 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:45.393299 (+ 529us) server_negotiation.cc:299] Negotiation successful
0811 05:13:45.393433 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 05:13:45.394407 17673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:45.379792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.129:50159)
0811 05:13:45.380239 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:45.380282 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:45.381490 (+ 1208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:45.382069 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:45.382076 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:45.382427 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:45.382887 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:45.382900 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:45.383864 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:45.383873 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:45.385110 (+ 1237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:45.385124 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:45.386008 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:45.386017 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:45.386277 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:45.387033 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:45.387058 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:45.389297 (+ 2239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:45.391437 (+ 2140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:45.391444 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:45.391459 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:45.391752 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:45.392080 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:45.392085 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:45.392087 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:45.392246 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:45.392769 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:45.392777 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:45.393036 (+ 259us) client_negotiation.cc:769] Sending connection context
0811 05:13:45.393234 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 05:13:45.393563 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":114,"threads_started":1}
WARNING: no policy specified for oryx/127.7.67.130@KRBTEST.COM; defaulting to no policy
I20260811 05:13:45.395865 17671 heartbeater.cc:347] Connected to a master server at 127.7.67.190:45385
I20260811 05:13:45.396126 17671 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:45.396711 17671 heartbeater.cc:511] Master 127.7.67.190:45385 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.130@KRBTEST.COM" created.
I20260811 05:13:45.398360 17361 ts_manager.cc:194] Registered new tserver with Master: d2e72a180c5e43a5ba0728584cc25be4 (127.7.67.129:36481)
I20260811 05:13:45.399552 17361 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.67.129@KRBTEST.COM'} at 127.7.67.129:50159
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.130.keytab.
Entry for principal oryx/127.7.67.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.130.keytab.
I20260811 05:13:45.416152 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:45385
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.130.keytab
--principal=oryx/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:45919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:45.529150 17677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:45.529398 17677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:45.529438 17677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:45.533401 17677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:45.533450 17677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:45.533504 17677 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:13:45.538627 17677 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:45919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.130.keytab
--principal=oryx/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:45385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:45.539569 17677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:45.540499 17677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:45.550405 17684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:45.550433 17689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:45.550652 17677 server_base.cc:1034] running on GCE node
W20260811 05:13:45.550824 17687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:45.551229 17677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:45.551849 17677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:45.553053 17677 hybrid_clock.cc:648] HybridClock initialized: now 1786425225553019 us; error 45 us; skew 500 ppm
Aug 11 05:13:45 dist-test-slave-04bb krb5kdc[17292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425225, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:45.555891 17677 init.cc:377] Logged in from keytab as oryx/127.7.67.130@KRBTEST.COM (short username oryx)
I20260811 05:13:45.556836 17677 webserver.cc:460] Webserver started at http://127.7.67.130:41043/ using document root <none> and password file <none>
I20260811 05:13:45.557376 17677 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:45.557439 17677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:45.557606 17677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:45.559734 17677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "82b986efd42f463ba147b7067465ac5e"
format_stamp: "Formatted at 2026-08-11 05:13:45 on dist-test-slave-04bb"
server_key: "c9af80dea12b953e678c67f86c8b7c6c"
server_key_iv: "a6f852ed55e8c41cbd3cef15cc77f489"
server_key_version: "encryptionkey@0"
I20260811 05:13:45.560294 17677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "82b986efd42f463ba147b7067465ac5e"
format_stamp: "Formatted at 2026-08-11 05:13:45 on dist-test-slave-04bb"
server_key: "c9af80dea12b953e678c67f86c8b7c6c"
server_key_iv: "a6f852ed55e8c41cbd3cef15cc77f489"
server_key_version: "encryptionkey@0"
I20260811 05:13:45.564038 17677 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 05:13:45.566495 17697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:45.567718 17677 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:13:45.567842 17677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "82b986efd42f463ba147b7067465ac5e"
format_stamp: "Formatted at 2026-08-11 05:13:45 on dist-test-slave-04bb"
server_key: "c9af80dea12b953e678c67f86c8b7c6c"
server_key_iv: "a6f852ed55e8c41cbd3cef15cc77f489"
server_key_version: "encryptionkey@0"
I20260811 05:13:45.567953 17677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:45.586530 17677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:45.589962 17677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:45.590173 17677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:45.590778 17677 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:45.591689 17677 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:45.591737 17677 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:45.591779 17677 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:45.591828 17677 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:45.601141 17677 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:40403
I20260811 05:13:45.601179 17862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:40403 every 8 connection(s)
I20260811 05:13:45.602049 17677 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:45.602933 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 17677
I20260811 05:13:45.603048 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:13:45.603339 7438 external_mini_cluster.cc:1428] Setting key e385aaf48b01bf144da64dd246a15646
Aug 11 05:13:45 dist-test-slave-04bb krb5kdc[17292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425225, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.7.67.131@KRBTEST.COM; defaulting to no policy
I20260811 05:13:45.617653 17429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:45.604683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:47917 (local address 127.7.67.190:45385)
0811 05:13:45.604839 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:45.604844 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:45.605247 (+ 403us) server_negotiation.cc:407] Connection header received
0811 05:13:45.606527 (+ 1280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:45.606530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:45.606582 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:45.606663 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:45.607885 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:45.608787 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:45.610093 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:45.610515 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:45.610647 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:45.613535 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:45.613539 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:45.613542 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:45.613571 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:45.615361 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:45.616085 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:45.616088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:45.616089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:45.616175 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:45.616571 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:45.616576 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:45.616581 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:45.616789 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:45.616870 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:45.617423 (+ 553us) server_negotiation.cc:299] Negotiation successful
0811 05:13:45.617527 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 05:13:45.618480 17869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:45.604429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.130:47917)
0811 05:13:45.605094 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:45.605136 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:45.606326 (+ 1190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:45.606839 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:45.606846 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:45.607244 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:45.607726 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:45.607740 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:45.608940 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:45.608943 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:45.609955 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:45.609966 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:45.610668 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:45.610671 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:45.610814 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:45.611523 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:45.611541 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:45.613344 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:45.615494 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:45.615503 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:45.615523 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:45.615937 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:45.616255 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:45.616276 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:45.616278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:45.616433 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:45.616909 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:45.616916 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:45.617161 (+ 245us) client_negotiation.cc:769] Sending connection context
0811 05:13:45.617353 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 05:13:45.617615 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":140,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.131@KRBTEST.COM" created.
I20260811 05:13:45.619907 17865 heartbeater.cc:347] Connected to a master server at 127.7.67.190:45385
I20260811 05:13:45.620213 17865 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:45.620860 17865 heartbeater.cc:511] Master 127.7.67.190:45385 requested a full tablet report, sending...
I20260811 05:13:45.622125 17361 ts_manager.cc:194] Registered new tserver with Master: 82b986efd42f463ba147b7067465ac5e (127.7.67.130:40403)
I20260811 05:13:45.623049 17361 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.67.130@KRBTEST.COM'} at 127.7.67.130:47917
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.131.keytab.
Entry for principal oryx/127.7.67.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.131.keytab.
I20260811 05:13:45.635886 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:45385
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.131.keytab
--principal=oryx/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:45919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:45.752074 17874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:45.752319 17874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:45.752362 17874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:45.756295 17874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:45.756348 17874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:45.756404 17874 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:13:45.761708 17874 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:45919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.131.keytab
--principal=oryx/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:45385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:45.762724 17874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:45.763700 17874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:45.773319 17891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:45.773331 17885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:45.773335 17887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:45.773399 17874 server_base.cc:1034] running on GCE node
I20260811 05:13:45.773984 17874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:45.774609 17874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:45.775771 17874 hybrid_clock.cc:648] HybridClock initialized: now 1786425225775757 us; error 26 us; skew 500 ppm
Aug 11 05:13:45 dist-test-slave-04bb krb5kdc[17292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425225, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:45.778528 17874 init.cc:377] Logged in from keytab as oryx/127.7.67.131@KRBTEST.COM (short username oryx)
I20260811 05:13:45.779369 17874 webserver.cc:460] Webserver started at http://127.7.67.131:34601/ using document root <none> and password file <none>
I20260811 05:13:45.779888 17874 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:45.779947 17874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:45.780109 17874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:45.782173 17874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "32e63505878c446594d70821360ed6c5"
format_stamp: "Formatted at 2026-08-11 05:13:45 on dist-test-slave-04bb"
server_key: "9ef99922e2724690e2dae84ea665c2e4"
server_key_iv: "b3d14fcca1d71a56ba54c1684604f392"
server_key_version: "encryptionkey@0"
I20260811 05:13:45.782703 17874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "32e63505878c446594d70821360ed6c5"
format_stamp: "Formatted at 2026-08-11 05:13:45 on dist-test-slave-04bb"
server_key: "9ef99922e2724690e2dae84ea665c2e4"
server_key_iv: "b3d14fcca1d71a56ba54c1684604f392"
server_key_version: "encryptionkey@0"
I20260811 05:13:45.786427 17874 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:13:45.789001 17902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:45.790295 17874 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:13:45.790441 17874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "32e63505878c446594d70821360ed6c5"
format_stamp: "Formatted at 2026-08-11 05:13:45 on dist-test-slave-04bb"
server_key: "9ef99922e2724690e2dae84ea665c2e4"
server_key_iv: "b3d14fcca1d71a56ba54c1684604f392"
server_key_version: "encryptionkey@0"
I20260811 05:13:45.790555 17874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:45.805539 17874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:45.808894 17874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:45.809165 17874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:45.809991 17874 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:45.810817 17874 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:45.810864 17874 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:45.810904 17874 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:45.810920 17874 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:45.820077 17874 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:43549
I20260811 05:13:45.820159 18064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:43549 every 8 connection(s)
I20260811 05:13:45.821035 17874 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:13:45.822976 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 17874
I20260811 05:13:45.823098 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:13:45.823426 7438 external_mini_cluster.cc:1428] Setting key b4d3b308c8586cbac8f0c2648c4fe8ce
Aug 11 05:13:45 dist-test-slave-04bb krb5kdc[17292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425225, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM
I20260811 05:13:45.834661 17429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:45.822911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:45047 (local address 127.7.67.190:45385)
0811 05:13:45.823055 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:45.823062 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:45.824202 (+ 1140us) server_negotiation.cc:407] Connection header received
0811 05:13:45.825743 (+ 1541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:45.825746 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:45.825792 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:45.825864 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:45.826949 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:45.827476 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:45.828639 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:45.828861 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:45.828942 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:45.831381 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:45.831384 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:45.831386 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:45.831408 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:45.832747 (+ 1339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:45.833272 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:45.833276 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:45.833277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:45.833361 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:45.833672 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:45.833675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:45.833676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:45.833875 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:45.833981 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:45.834441 (+ 460us) server_negotiation.cc:299] Negotiation successful
0811 05:13:45.834543 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 05:13:45.835523 18067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:45.823436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.131:45047)
0811 05:13:45.824025 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:45.824069 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:45.825519 (+ 1450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:45.825996 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:45.826003 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:45.826350 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:45.826819 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:45.826831 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:45.827585 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:45.827588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:45.828464 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:45.828472 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:45.828991 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:45.828994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:45.829130 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:45.829791 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:45.829807 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:45.831231 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:45.832870 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:45.832875 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:45.832886 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:45.833154 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:45.833452 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:45.833454 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:45.833455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:45.833586 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:45.834001 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:45.834006 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:45.834228 (+ 222us) client_negotiation.cc:769] Sending connection context
0811 05:13:45.834387 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 05:13:45.834638 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":153,"threads_started":1}
I20260811 05:13:45.836791 18065 heartbeater.cc:347] Connected to a master server at 127.7.67.190:45385
I20260811 05:13:45.837019 18065 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:45.837528 18065 heartbeater.cc:511] Master 127.7.67.190:45385 requested a full tablet report, sending...
I20260811 05:13:45.838557 17360 ts_manager.cc:194] Registered new tserver with Master: 32e63505878c446594d70821360ed6c5 (127.7.67.131:43549)
I20260811 05:13:45.839145 17360 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.67.131@KRBTEST.COM'} at 127.7.67.131:45047
I20260811 05:13:45.849042 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.67.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.189.keytab.
Entry for principal oryx/127.7.67.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.189.keytab.
I20260811 05:13:45.879273 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.189:40365
--webserver_interface=127.7.67.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.189.keytab
--principal=oryx/127.7.67.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:45919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:45385,127.7.67.189:40365
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:45.996176 18070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:45.996438 18070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:45.996477 18070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:46.000491 18070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:46.000567 18070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:46.000587 18070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:46.000603 18070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:46.000619 18070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:46.005963 18070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:45919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:45385,127.7.67.189:40365
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.189.keytab
--principal=oryx/127.7.67.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.189:40365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.67.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
W20260811 05:13:46.006187 18070 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.7.67.190:45385,127.7.67.189:40365'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 05:13:46.007102 18070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:46.008221 18070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:46.016861 18085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:46.016860 18083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:46.016942 18089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:46.016994 18070 server_base.cc:1034] running on GCE node
I20260811 05:13:46.017642 18070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:46.018746 18070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:46.019935 18070 hybrid_clock.cc:648] HybridClock initialized: now 1786425226019905 us; error 40 us; skew 500 ppm
Aug 11 05:13:46 dist-test-slave-04bb krb5kdc[17292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425226, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:46.023072 18070 init.cc:377] Logged in from keytab as oryx/127.7.67.189@KRBTEST.COM (short username oryx)
I20260811 05:13:46.023957 18070 webserver.cc:460] Webserver started at http://127.7.67.189:45751/ using document root <none> and password file <none>
I20260811 05:13:46.024600 18070 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:46.024703 18070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:46.024930 18070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:46.027050 18070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/data/instance:
uuid: "ba069f98f23c4f3b9125de6627e8992d"
format_stamp: "Formatted at 2026-08-11 05:13:46 on dist-test-slave-04bb"
server_key: "159ab97640679a4107b7a1173e15408e"
server_key_iv: "2d5d87b3d4ef8243e7a2396156c87c33"
server_key_version: "encryptionkey@0"
I20260811 05:13:46.027634 18070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/wal/instance:
uuid: "ba069f98f23c4f3b9125de6627e8992d"
format_stamp: "Formatted at 2026-08-11 05:13:46 on dist-test-slave-04bb"
server_key: "159ab97640679a4107b7a1173e15408e"
server_key_iv: "2d5d87b3d4ef8243e7a2396156c87c33"
server_key_version: "encryptionkey@0"
I20260811 05:13:46.031512 18070 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 05:13:46.034348 18102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:46.035647 18070 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 05:13:46.035856 18070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/wal
uuid: "ba069f98f23c4f3b9125de6627e8992d"
format_stamp: "Formatted at 2026-08-11 05:13:46 on dist-test-slave-04bb"
server_key: "159ab97640679a4107b7a1173e15408e"
server_key_iv: "2d5d87b3d4ef8243e7a2396156c87c33"
server_key_version: "encryptionkey@0"
I20260811 05:13:46.035984 18070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:46.055685 18070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:46.058977 18070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:46.059206 18070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:46.066164 18070 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.189:40365
I20260811 05:13:46.066532 18180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.189:40365 every 8 connection(s)
I20260811 05:13:46.067173 18070 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/data/info.pb
I20260811 05:13:46.070281 18181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:46.074105 18181 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:13:46.076961 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 18070
Aug 11 05:13:46 dist-test-slave-04bb krb5kdc[17292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425226, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.189@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM
Aug 11 05:13:46 dist-test-slave-04bb krb5kdc[17292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.67.189@KRBTEST.COM
I20260811 05:13:46.088876 17429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.075933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57326 (local address 127.7.67.190:45385)
0811 05:13:46.076094 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:46.076099 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:46.077111 (+ 1012us) server_negotiation.cc:407] Connection header received
0811 05:13:46.078345 (+ 1234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:46.078348 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:46.078411 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:46.078518 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:46.079485 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.080019 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.081099 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.081326 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.081439 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.084283 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:46.084290 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:46.084295 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:46.084329 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:46.086217 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.086742 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.086748 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.086752 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.086849 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.087401 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.087406 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.087407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.087748 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:46.087903 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:46.088458 (+ 555us) server_negotiation.cc:299] Negotiation successful
0811 05:13:46.088668 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 05:13:46.089681 18185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.076348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.0.0.1:57326)
0811 05:13:46.076928 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:46.076999 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:46.078080 (+ 1081us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:46.078604 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:46.078615 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:46.078957 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:46.079338 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.079350 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.080160 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.080163 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.080973 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.080983 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.081490 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.081494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.081657 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.082266 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:46.082289 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:46.084133 (+ 1844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:46.086341 (+ 2208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.086348 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.086356 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.086615 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.087024 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.087028 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.087029 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.087221 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.087901 (+ 680us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:46.087906 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:46.088138 (+ 232us) client_negotiation.cc:769] Sending connection context
0811 05:13:46.088377 (+ 239us) client_negotiation.cc:240] Negotiation successful
0811 05:13:46.088761 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":167,"threads_started":1}
I20260811 05:13:46.089690 18188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.077845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44240 (local address 127.7.67.189:40365)
0811 05:13:46.078260 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:46.078275 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:46.078322 (+ 47us) server_negotiation.cc:407] Connection header received
0811 05:13:46.078457 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:46.078461 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:46.078616 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:46.078800 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:46.079598 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.080441 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.081301 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.081662 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.081779 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.084936 (+ 3157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:46.084945 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:46.084958 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:46.084983 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:46.086856 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.087411 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.087417 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.087419 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.087512 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.087830 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.087846 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.087854 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.088154 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:46.088407 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:46.088671 (+ 264us) server_negotiation.cc:299] Negotiation successful
0811 05:13:46.088824 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"spinlock_wait_cycles":2304,"thread_start_us":141,"threads_started":1}
I20260811 05:13:46.091230 18181 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:13:46 dist-test-slave-04bb krb5kdc[17292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425226, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.189@KRBTEST.COM for oryx/127.7.67.189@KRBTEST.COM
I20260811 05:13:46.102833 18188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.092060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44244 (local address 127.7.67.189:40365)
0811 05:13:46.092405 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:46.092409 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:46.092420 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:13:46.092468 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:46.092472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:46.092510 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:46.092600 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:46.093252 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.094215 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.095653 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.096727 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.096844 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.099390 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:46.099394 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:46.099397 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:46.099421 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:46.101060 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.101550 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.101557 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.101559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.101650 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.101960 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.101963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.101965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.102140 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:46.102256 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:46.102522 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 05:13:46.102681 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":257}
I20260811 05:13:46.102833 18185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.091807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:40365 (local address 127.0.0.1:44244)
0811 05:13:46.092096 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:46.092109 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:46.092205 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:46.092711 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:46.092716 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:46.092946 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:46.093098 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.093108 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.094373 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.094378 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.095510 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.095525 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.096888 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.096893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.097072 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.097702 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:46.097719 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:46.099242 (+ 1523us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:46.101200 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.101207 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.101211 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.101414 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.101761 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.101764 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.101767 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.101855 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.102258 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:46.102262 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:46.102357 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 05:13:46.102474 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 05:13:46.102620 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203}
I20260811 05:13:46.106168 18181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d: Bootstrap starting.
I20260811 05:13:46.108954 18181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:46.109790 18181 log.cc:824] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:46.111600 18181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d: No bootstrap required, opened a new log
I20260811 05:13:46.114501 18181 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } }
I20260811 05:13:46.114686 18181 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:46.114718 18181 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba069f98f23c4f3b9125de6627e8992d, State: Initialized, Role: FOLLOWER
I20260811 05:13:46.115077 18181 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } }
I20260811 05:13:46.115483 18190 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } } }
I20260811 05:13:46.115619 18190 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:13:46.116228 18181 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [sys.catalog]: configured and running, proceeding with master startup.
W20260811 05:13:46.121141 18209 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:13:46.121243 18209 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:13:46.123036 18070 master_runner.cc:428] Detected that this master ba069f98f23c4f3b9125de6627e8992d is joining an existing cluster
I20260811 05:13:46.123106 18070 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.67.189:40365
I20260811 05:13:46.133190 18227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.125091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:40365 (local address 127.0.0.1:44256)
0811 05:13:46.125483 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:46.125495 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:46.125596 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:46.125825 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:46.125828 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:46.126035 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:46.126131 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.126136 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.127160 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.127164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.128197 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.128209 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.128711 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.128716 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.128821 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.129341 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:46.129356 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:46.129826 (+ 470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:46.131523 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.131527 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.131529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.131798 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.132072 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.132076 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.132078 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.132180 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.132579 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:46.132583 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:46.132701 (+ 118us) client_negotiation.cc:769] Sending connection context
0811 05:13:46.132861 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 05:13:46.132999 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":138,"threads_started":1}
I20260811 05:13:46.133304 18188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.125511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44256 (local address 127.7.67.189:40365)
0811 05:13:46.125635 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:46.125641 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:46.125655 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:13:46.125685 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:46.125688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:46.125722 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:46.125812 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:46.126249 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.127021 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.128318 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.128582 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.128674 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.129979 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:46.129982 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:46.129984 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:46.130004 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:46.131411 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.131901 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.131903 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.131905 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.131976 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.132275 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.132277 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.132278 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.132451 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:46.132594 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:46.132969 (+ 375us) server_negotiation.cc:299] Negotiation successful
0811 05:13:46.133100 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 05:13:46.140877 18226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.125201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.0.0.1:57342)
0811 05:13:46.132690 (+ 7489us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:46.132704 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:46.132814 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:46.133236 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:46.133240 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:46.133471 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:46.133947 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.133958 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.135016 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.135019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.135668 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.135676 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.136280 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.136285 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.136373 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.136893 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:46.136905 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:46.137222 (+ 317us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:46.139219 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.139222 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.139224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.139443 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.139737 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.139740 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.139741 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.139822 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.140273 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:46.140277 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:46.140388 (+ 111us) client_negotiation.cc:769] Sending connection context
0811 05:13:46.140509 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 05:13:46.140674 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7406,"thread_start_us":107,"threads_started":1}
I20260811 05:13:46.140900 17429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.125124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57342 (local address 127.7.67.190:45385)
0811 05:13:46.125275 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:46.125280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:46.132870 (+ 7590us) server_negotiation.cc:407] Connection header received
0811 05:13:46.132929 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:46.132932 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:46.132986 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:46.133085 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:46.134093 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.134902 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.135797 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.136140 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.136247 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.137365 (+ 1118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:46.137369 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:46.137372 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:46.137399 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:46.139074 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.139542 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.139546 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.139547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.139638 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.139940 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.139944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.139946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.140161 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:46.140284 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:46.140595 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 05:13:46.140736 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 05:13:46 dist-test-slave-04bb krb5kdc[17292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425225, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.190@KRBTEST.COM for oryx/127.7.67.189@KRBTEST.COM
WARNING: no policy specified for oryx/127.7.67.188@KRBTEST.COM; defaulting to no policy
I20260811 05:13:46.157372 18188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.147279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44258 (local address 127.7.67.189:40365)
0811 05:13:46.147395 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:46.147399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:46.147413 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:13:46.147480 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:46.147485 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:46.147533 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:46.147644 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:46.148336 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.149128 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.150434 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.150767 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.150838 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.153519 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:46.153525 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:46.153531 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:46.153564 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:46.155363 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.155980 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.155985 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.155986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.156074 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.156416 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.156419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.156421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.156651 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:46.156780 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:46.157178 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 05:13:46.157266 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 05:13:46.157574 18229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.146532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:40365 (local address 127.0.0.1:44258)
0811 05:13:46.146893 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:46.146945 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:46.147073 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:46.147689 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:46.147697 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:46.148074 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:46.148205 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.148211 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.149260 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.149263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.150279 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.150290 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.150981 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.150985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.151109 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.151604 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:46.151619 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:46.153366 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:46.155496 (+ 2130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.155506 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.155514 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.155857 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.156194 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.156197 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.156200 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.156302 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.156780 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:46.156787 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:46.156928 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 05:13:46.157104 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 05:13:46.157306 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":105,"threads_started":1}
I20260811 05:13:46.159173 17360 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.67.189:40365: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d"
member_type: NON_VOTER
last_known_addr {
host: "127.7.67.189"
port: 40365
}
attrs {
promote: true
}
}
dest_uuid: "95667e6974cb44ba9111427a619ddcef"
cas_config_opid_index: -1
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.188@KRBTEST.COM" created.
I20260811 05:13:46.159986 17360 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: NON_VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: true } }
I20260811 05:13:46.161933 18231 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER ba069f98f23c4f3b9125de6627e8992d (127.7.67.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: NON_VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: true } } }
I20260811 05:13:46.162851 18142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:46.163914 18231 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.67.189:40365
I20260811 05:13:46.164374 18142 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 1 FOLLOWER]: Refusing update from remote peer 95667e6974cb44ba9111427a619ddcef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 05:13:46.164397 18070 master.cc:537] Master@127.7.67.189:40365 shutting down...
I20260811 05:13:46.164438 18231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "95667e6974cb44ba9111427a619ddcef" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: NON_VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: true } } }
I20260811 05:13:46.164546 18231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:46.165144 18232 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: NON_VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 05:13:46.167377 17342 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 05:13:46.168496 17342 consensus_peers.cc:487] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef -> Peer ba069f98f23c4f3b9125de6627e8992d (127.7.67.189:40365): Couldn't send request to peer ba069f98f23c4f3b9125de6627e8992d. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 05:13:46.170373 18070 raft_consensus.cc:2229] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:46.170737 18070 raft_consensus.cc:2258] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:46.170915 18070 tablet_replica.cc:330] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.188.keytab.
Entry for principal oryx/127.7.67.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.188.keytab.
I20260811 05:13:46.179814 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.188:36501
--webserver_interface=127.7.67.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.188.keytab
--principal=oryx/127.7.67.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:45919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:45385,127.7.67.189:40365,127.7.67.188:36501
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:13:46.186980 18070 master.cc:552] Master@127.7.67.189:40365 shutdown complete.
I20260811 05:13:46.187047 18070 master_runner.cc:305] Clearing existing system tablet
I20260811 05:13:46.187340 18070 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:46.187410 18070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:46.189106 18070 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.003s sys 0.000s
I20260811 05:13:46.190196 18239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:46.190435 18070 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:46.190557 18070 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:46.190591 18070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/wal
uuid: "ba069f98f23c4f3b9125de6627e8992d"
format_stamp: "Formatted at 2026-08-11 05:13:46 on dist-test-slave-04bb"
server_key: "159ab97640679a4107b7a1173e15408e"
server_key_iv: "2d5d87b3d4ef8243e7a2396156c87c33"
server_key_version: "encryptionkey@0"
I20260811 05:13:46.192955 18070 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:46.195626 18070 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 05:13:46.195739 18070 log.cc:1196] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 05:13:46.196012 18070 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d: Deleting consensus metadata
I20260811 05:13:46.196198 18070 master_runner.cc:315] Copying system tablet from 127.7.67.190:45385
I20260811 05:13:46.197551 18070 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d: tablet copy: Beginning tablet copy session from remote peer at address 127.7.67.190:45385
I20260811 05:13:46.207134 18247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.198169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.0.0.1:57348)
0811 05:13:46.198472 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:46.198487 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:46.198604 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:46.198894 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:46.198897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:46.199111 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:46.199241 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.199248 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.200412 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.200416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.201382 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.201393 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.202046 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.202050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.202157 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.202804 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:46.202825 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:46.203256 (+ 431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:46.205448 (+ 2192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.205451 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.205453 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.205692 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.206003 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.206005 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.206007 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.206101 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.206566 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:46.206568 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:46.206670 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 05:13:46.206787 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 05:13:46.206930 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":82,"threads_started":1}
I20260811 05:13:46.207177 17429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.198288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57348 (local address 127.7.67.190:45385)
0811 05:13:46.198423 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:46.198429 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:46.198576 (+ 147us) server_negotiation.cc:407] Connection header received
0811 05:13:46.198699 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:46.198703 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:46.198759 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:46.198864 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:46.199374 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.200282 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.201516 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.201916 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.202053 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.203422 (+ 1369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:46.203426 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:46.203429 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:46.203461 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:46.205325 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.205801 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.205805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.205807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.205900 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.206210 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.206214 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.206216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.206457 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:46.206569 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:46.206870 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 05:13:46.207008 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 05:13:46.208159 17391 tablet_copy_service.cc:140] P 95667e6974cb44ba9111427a619ddcef: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer ba069f98f23c4f3b9125de6627e8992d ({username='oryx', principal='oryx/127.7.67.189@KRBTEST.COM'} at 127.0.0.1:57348)
I20260811 05:13:46.208351 17391 tablet_copy_service.cc:161] P 95667e6974cb44ba9111427a619ddcef: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer ba069f98f23c4f3b9125de6627e8992d at {username='oryx', principal='oryx/127.7.67.189@KRBTEST.COM'} at 127.0.0.1:57348: session id = ba069f98f23c4f3b9125de6627e8992d-00000000000000000000000000000000
I20260811 05:13:46.210273 17391 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 05:13:46.212030 18070 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 05:13:46.214378 18070 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d: tablet copy: Starting download of 0 data blocks...
I20260811 05:13:46.214570 18070 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d: tablet copy: Starting download of 1 WAL segments...
I20260811 05:13:46.216595 18070 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 05:13:46.218701 17391 tablet_copy_service.cc:342] P 95667e6974cb44ba9111427a619ddcef: Request end of tablet copy session ba069f98f23c4f3b9125de6627e8992d-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.67.189@KRBTEST.COM'} at 127.0.0.1:57348
I20260811 05:13:46.218833 17391 tablet_copy_service.cc:433] P 95667e6974cb44ba9111427a619ddcef: ending tablet copy session ba069f98f23c4f3b9125de6627e8992d-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer ba069f98f23c4f3b9125de6627e8992d
I20260811 05:13:46.226668 18070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:46.228574 18248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:46.228627 18254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:46.228754 18250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:46.229128 18070 server_base.cc:1034] running on GCE node
I20260811 05:13:46.229274 18070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:46.229586 18070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:46.230734 18070 hybrid_clock.cc:648] HybridClock initialized: now 1786425226230718 us; error 36 us; skew 500 ppm
Aug 11 05:13:46 dist-test-slave-04bb krb5kdc[17292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425226, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:46.232434 18070 init.cc:377] Logged in from keytab as oryx/127.7.67.189@KRBTEST.COM (short username oryx)
I20260811 05:13:46.232968 18070 webserver.cc:460] Webserver started at http://127.7.67.189:36055/ using document root <none> and password file <none>
I20260811 05:13:46.233172 18070 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:46.233220 18070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:46.234771 18070 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 05:13:46.235948 18263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:46.236239 18070 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:46.236325 18070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/wal
uuid: "ba069f98f23c4f3b9125de6627e8992d"
format_stamp: "Formatted at 2026-08-11 05:13:46 on dist-test-slave-04bb"
server_key: "159ab97640679a4107b7a1173e15408e"
server_key_iv: "2d5d87b3d4ef8243e7a2396156c87c33"
server_key_version: "encryptionkey@0"
I20260811 05:13:46.236413 18070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:46.273347 18070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:46.275894 18070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:46.282927 18070 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.189:40365
I20260811 05:13:46.283044 18340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.189:40365 every 8 connection(s)
I20260811 05:13:46.283638 18070 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-1/data/info.pb
I20260811 05:13:46.285619 18341 sys_catalog.cc:263] Verifying existing consensus state
I20260811 05:13:46.286711 18341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d: Bootstrap starting.
I20260811 05:13:46.298329 18341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:13:46.298638 18341 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d: Bootstrap complete.
I20260811 05:13:46.299227 18341 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: NON_VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: true } }
I20260811 05:13:46.299366 18341 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: ba069f98f23c4f3b9125de6627e8992d, State: Initialized, Role: LEARNER
I20260811 05:13:46.299490 18341 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: NON_VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: true } }
I20260811 05:13:46.299849 18344 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: NON_VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: true } } }
I20260811 05:13:46.299980 18341 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:46.300063 18344 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [sys.catalog]: This master's current role is: LEARNER
I20260811 05:13:46.303054 18367 catalog_manager.cc:1259] Loaded cluster ID: e9f81071b52b4f468d783cb68b56dd85
I20260811 05:13:46.303184 18367 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d: loading cluster ID for follower catalog manager: success
I20260811 05:13:46.305583 18367 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d: acquiring CA information for follower catalog manager: success
I20260811 05:13:46.306985 18367 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 05:13:46.316493 18237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:46.316792 18237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:46.316883 18237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:46.321110 18237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:46.321178 18237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:46.321223 18237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:46.321278 18237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:46.321322 18237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:46.326676 18237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:45919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:45385,127.7.67.189:40365,127.7.67.188:36501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.188.keytab
--principal=oryx/127.7.67.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.188:36501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.67.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:46.327672 18237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:46.328732 18237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:46.337277 18376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:46.337281 18372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:46.337548 18237 server_base.cc:1034] running on GCE node
W20260811 05:13:46.337561 18373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:46.338141 18237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:46.339223 18237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:46.340399 18237 hybrid_clock.cc:648] HybridClock initialized: now 1786425226340381 us; error 29 us; skew 500 ppm
Aug 11 05:13:46 dist-test-slave-04bb krb5kdc[17292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425226, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:46.343227 18237 init.cc:377] Logged in from keytab as oryx/127.7.67.188@KRBTEST.COM (short username oryx)
I20260811 05:13:46.344292 18237 webserver.cc:460] Webserver started at http://127.7.67.188:37023/ using document root <none> and password file <none>
I20260811 05:13:46.344928 18237 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:46.345024 18237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:46.345240 18237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:46.347273 18237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/data/instance:
uuid: "6f60f0e75bad4042a07087df578dfbdb"
format_stamp: "Formatted at 2026-08-11 05:13:46 on dist-test-slave-04bb"
server_key: "f146197f431ad270f46faa3fc0b22e2b"
server_key_iv: "239b7119192aa69a5de28b45a29dc2d9"
server_key_version: "encryptionkey@0"
I20260811 05:13:46.347852 18237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/wal/instance:
uuid: "6f60f0e75bad4042a07087df578dfbdb"
format_stamp: "Formatted at 2026-08-11 05:13:46 on dist-test-slave-04bb"
server_key: "f146197f431ad270f46faa3fc0b22e2b"
server_key_iv: "239b7119192aa69a5de28b45a29dc2d9"
server_key_version: "encryptionkey@0"
I20260811 05:13:46.351477 18237 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 05:13:46.353926 18388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:46.355129 18237 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:46.355269 18237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/wal
uuid: "6f60f0e75bad4042a07087df578dfbdb"
format_stamp: "Formatted at 2026-08-11 05:13:46 on dist-test-slave-04bb"
server_key: "f146197f431ad270f46faa3fc0b22e2b"
server_key_iv: "239b7119192aa69a5de28b45a29dc2d9"
server_key_version: "encryptionkey@0"
I20260811 05:13:46.355396 18237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:46.385560 18237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:46.389169 18237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:46.389405 18237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:46.398636 18237 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.188:36501
I20260811 05:13:46.398902 18461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.188:36501 every 8 connection(s)
I20260811 05:13:46.399664 18237 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/data/info.pb
I20260811 05:13:46.402037 17671 heartbeater.cc:503] Master 127.7.67.190:45385 was elected leader, sending a full tablet report...
I20260811 05:13:46.403048 18462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:46.406970 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 18237
I20260811 05:13:46.406862 18462 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:13:46 dist-test-slave-04bb krb5kdc[17292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425226, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.188@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM
Aug 11 05:13:46 dist-test-slave-04bb krb5kdc[17292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425224, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.67.188@KRBTEST.COM
I20260811 05:13:46.420437 17429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.409595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57352 (local address 127.7.67.190:45385)
0811 05:13:46.409720 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:46.409725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:46.409951 (+ 226us) server_negotiation.cc:407] Connection header received
0811 05:13:46.410282 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:46.410285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:46.410340 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:46.410433 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:46.411355 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.411865 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.413016 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.413220 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.413303 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.416086 (+ 2783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:46.416090 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:46.416093 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:46.416137 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:46.417919 (+ 1782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.418497 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.418500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.418502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.418606 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.419083 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.419087 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.419088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.419370 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:46.419474 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:46.420131 (+ 657us) server_negotiation.cc:299] Negotiation successful
0811 05:13:46.420319 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:13:46.421525 18467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.408395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55754 (local address 127.7.67.188:36501)
0811 05:13:46.408819 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:46.408836 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:46.408878 (+ 42us) server_negotiation.cc:407] Connection header received
0811 05:13:46.409695 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:46.409721 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:46.410087 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:46.410318 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:46.411086 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.412105 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.412978 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.413425 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.413613 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.416613 (+ 3000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:46.416618 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:46.416627 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:46.416670 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:46.418491 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.419095 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.419100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.419102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.419194 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.419508 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.419513 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.419516 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.419861 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:46.420012 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:46.420899 (+ 887us) server_negotiation.cc:299] Negotiation successful
0811 05:13:46.421040 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":109,"server-negotiator.queue_time_us":292,"spinlock_wait_cycles":2176,"thread_start_us":168,"threads_started":1}
I20260811 05:13:46.421526 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.409379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.0.0.1:57352)
0811 05:13:46.409806 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:46.409853 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:46.410072 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:46.410500 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:46.410505 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:46.410892 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:46.411206 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.411220 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.411992 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.411995 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.412870 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.412879 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.413358 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.413362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.413611 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.414377 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:46.414392 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:46.415920 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:46.418037 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.418045 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.418052 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.418396 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.418734 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.418740 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.418745 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.418958 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.419487 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:46.419497 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:46.419857 (+ 360us) client_negotiation.cc:769] Sending connection context
0811 05:13:46.420064 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 05:13:46.420441 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":28,"spinlock_wait_cycles":2944,"thread_start_us":156,"threads_started":1}
I20260811 05:13:46.423662 18462 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:13:46 dist-test-slave-04bb krb5kdc[17292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425226, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.188@KRBTEST.COM for oryx/127.7.67.189@KRBTEST.COM
I20260811 05:13:46.431783 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.424108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:40365 (local address 127.0.0.1:44270)
0811 05:13:46.424272 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:46.424285 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:46.424352 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:46.424783 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:46.424785 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:46.424918 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:46.425010 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.425015 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.425874 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.425877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.426498 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.426507 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.426973 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.426977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.427073 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.427636 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:46.427648 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:46.428968 (+ 1320us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:46.430532 (+ 1564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.430535 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.430536 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.430740 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.430996 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.430998 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.431000 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.431075 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.431389 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:46.431392 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:46.431478 (+ 86us) client_negotiation.cc:769] Sending connection context
0811 05:13:46.431565 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 05:13:46.431662 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260811 05:13:46.431831 18469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.424272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44270 (local address 127.7.67.189:40365)
0811 05:13:46.424599 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:46.424602 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:46.424615 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:13:46.424650 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:46.424653 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:46.424691 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:46.424763 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:46.425115 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.425750 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.426605 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.426836 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.426915 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.429110 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:46.429113 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:46.429115 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:46.429136 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:46.430427 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.430839 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.430842 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.430843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.430913 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.431154 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.431156 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.431157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.431307 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:46.431378 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:46.431619 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 05:13:46.431707 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":114,"threads_started":1}
I20260811 05:13:46.432633 18462 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:13:46 dist-test-slave-04bb krb5kdc[17292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425226, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.188@KRBTEST.COM for oryx/127.7.67.188@KRBTEST.COM
I20260811 05:13:46.440595 18468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.433143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:36501 (local address 127.0.0.1:55760)
0811 05:13:46.433301 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:46.433315 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:46.433406 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:46.433630 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:46.433632 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:46.433783 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:46.433839 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.433844 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.434535 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.434538 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.435007 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.435013 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.435464 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.435467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.435581 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.436040 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:46.436052 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:46.437270 (+ 1218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:46.439123 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.439125 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.439127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.439348 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.439622 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.439627 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.439629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.439702 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.440052 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:46.440055 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:46.440135 (+ 80us) client_negotiation.cc:769] Sending connection context
0811 05:13:46.440235 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 05:13:46.440409 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":21}
I20260811 05:13:46.440604 18467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.433206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55760 (local address 127.7.67.188:36501)
0811 05:13:46.433318 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:46.433323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:46.433360 (+ 37us) server_negotiation.cc:407] Connection header received
0811 05:13:46.433498 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:46.433501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:46.433536 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:46.433619 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:46.433948 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.434423 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.435122 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.435360 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.435464 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.437421 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:46.437424 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:46.437426 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:46.437448 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:46.439022 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.439460 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.439463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.439464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.439530 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.439807 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.439812 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.439818 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.439958 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:46.440060 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:46.440292 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 05:13:46.440465 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":119,"server-negotiator.queue_time_us":46}
I20260811 05:13:46.444196 18462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb: Bootstrap starting.
I20260811 05:13:46.446516 18462 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:46.447188 18462 log.cc:824] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:46.449251 18462 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb: No bootstrap required, opened a new log
I20260811 05:13:46.452108 18462 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } }
I20260811 05:13:46.452301 18462 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:46.452334 18462 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f60f0e75bad4042a07087df578dfbdb, State: Initialized, Role: FOLLOWER
I20260811 05:13:46.452742 18462 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } }
I20260811 05:13:46.453166 18472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } } }
I20260811 05:13:46.453348 18472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:13:46.453873 18462 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [sys.catalog]: configured and running, proceeding with master startup.
W20260811 05:13:46.459118 18484 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:13:46.459224 18484 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:13:46.462749 18237 master_runner.cc:428] Detected that this master 6f60f0e75bad4042a07087df578dfbdb is joining an existing cluster
I20260811 05:13:46.462826 18237 master_runner.cc:432] Initiating AddMaster RPC to add 127.7.67.188:36501
I20260811 05:13:46.474973 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 17467
I20260811 05:13:46.512094 18467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.465795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55766 (local address 127.7.67.188:36501)
0811 05:13:46.465902 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:46.465908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:46.466459 (+ 551us) server_negotiation.cc:407] Connection header received
0811 05:13:46.466576 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:46.466582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:46.466635 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:46.466720 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:46.467355 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.468175 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.469442 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.469809 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.469883 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.471549 (+ 1666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:46.471554 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:46.471556 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:46.471581 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:46.509818 (+ 38237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.510389 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.510396 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.510401 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.510511 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.510866 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.510872 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.510877 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.511161 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:46.511247 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:46.511648 (+ 401us) server_negotiation.cc:299] Negotiation successful
0811 05:13:46.511815 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 05:13:46.512476 17429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.465347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57356 (local address 127.7.67.190:45385)
0811 05:13:46.465484 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:46.465490 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:46.465741 (+ 251us) server_negotiation.cc:407] Connection header received
0811 05:13:46.465926 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:46.465931 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:46.465994 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:46.466099 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:46.466683 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.467523 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.469640 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.469988 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.470064 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.508339 (+ 38275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:46.508344 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:46.508347 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:46.508389 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:46.510577 (+ 2188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.511117 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.511121 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.511123 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.511214 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.511527 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.511530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.511532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.511750 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:46.511827 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:46.512176 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 05:13:46.512315 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 05:13:46.512827 18497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.465777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:36501 (local address 127.0.0.1:55766)
0811 05:13:46.466363 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:46.466374 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:46.466496 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:46.466843 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:46.466846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:46.467095 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:46.467231 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.467242 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.468300 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.468303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.469281 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.469298 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.470054 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.470057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.470144 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.470801 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:46.470824 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:46.471369 (+ 545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:46.509961 (+ 38592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.509968 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.509973 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.510268 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.510637 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.510643 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.510648 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.510764 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.511315 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:46.511321 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:46.511460 (+ 139us) client_negotiation.cc:769] Sending connection context
0811 05:13:46.512367 (+ 907us) client_negotiation.cc:240] Negotiation successful
0811 05:13:46.512514 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"mutex_wait_us":3,"thread_start_us":92,"threads_started":1}
I20260811 05:13:46.513018 18495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.465263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.0.0.1:57356)
0811 05:13:46.465658 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:46.465673 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:46.465813 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:46.466125 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:46.466128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:46.466386 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:46.466533 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.466541 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.467669 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.467672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.469494 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.469505 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.470228 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.470233 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.470336 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.507546 (+ 37210us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:46.507573 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:46.508093 (+ 520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:46.510736 (+ 2643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.510740 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.510742 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.510993 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.511342 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.511345 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.511347 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.511427 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.511901 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:46.511904 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:46.511998 (+ 94us) client_negotiation.cc:769] Sending connection context
0811 05:13:46.512702 (+ 704us) client_negotiation.cc:240] Negotiation successful
0811 05:13:46.512846 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":61,"thread_start_us":108,"threads_started":1}
I20260811 05:13:46.515338 18469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.465425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44282 (local address 127.7.67.189:40365)
0811 05:13:46.465543 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:46.465548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:46.466817 (+ 1269us) server_negotiation.cc:407] Connection header received
0811 05:13:46.467005 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:46.467009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:46.467083 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:46.467144 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:46.467791 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.468686 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.470072 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.470852 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.470958 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.507925 (+ 36967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:46.507933 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:46.507938 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:46.507979 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:46.513221 (+ 5242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.513794 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.513798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.513800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.513893 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.514271 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.514275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.514277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.514533 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:46.514619 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:46.515016 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 05:13:46.515165 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 05:13:46.515920 18498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.465263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:40365 (local address 127.0.0.1:44282)
0811 05:13:46.466734 (+ 1471us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:46.466748 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:46.466877 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:46.467313 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:46.467319 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:46.467531 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:46.467640 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.467647 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.468834 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.468837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.469936 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.469945 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.471141 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.471144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.471254 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.471778 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:46.471788 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:46.507628 (+ 35840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:46.513395 (+ 5767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.513398 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.513400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.513655 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.514020 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.514023 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.514025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.514130 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.514705 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:46.514708 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:46.514795 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 05:13:46.515495 (+ 700us) client_negotiation.cc:240] Negotiation successful
0811 05:13:46.515668 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1398,"mutex_wait_us":805,"thread_start_us":110,"threads_started":1}
I20260811 05:13:46.519393 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:36481
--local_ip_for_outbound_sockets=127.7.67.129
--tserver_master_addrs=127.7.67.190:45385
--webserver_port=46679
--webserver_interface=127.7.67.129
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.129.keytab
--principal=oryx/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:45919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.67.190:45385,127.7.67.189:40365
--tserver_master_addrs=127.7.67.190:45385,127.7.67.189:40365,127.7.67.188:36501 with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:13:46 dist-test-slave-04bb krb5kdc[17292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425225, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.190@KRBTEST.COM for oryx/127.7.67.188@KRBTEST.COM
I20260811 05:13:46.527552 18467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.518300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55776 (local address 127.7.67.188:36501)
0811 05:13:46.518422 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:46.518425 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:46.518436 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:13:46.518494 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:46.518496 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:46.518565 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:46.518651 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:46.519319 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.520127 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.521424 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.521636 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.521761 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.523900 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:46.523904 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:46.523907 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:46.523957 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:46.525785 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.526259 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.526264 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.526266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.526353 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.526663 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.526668 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.526670 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.526855 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:46.526981 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:46.527283 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 05:13:46.527392 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:13:46.527554 18229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.517830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:36501 (local address 127.0.0.1:55776)
0811 05:13:46.517948 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:46.517963 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:46.518044 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:46.518832 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:46.518839 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:46.519044 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:46.519172 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.519179 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.520278 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.520285 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.521287 (+ 1002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.521299 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.521778 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.521783 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.521885 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.522385 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:46.522403 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:46.523755 (+ 1352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:46.525915 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.525920 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.525923 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.526141 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.526458 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.526462 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.526464 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.526560 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.526965 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:46.526968 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:46.527081 (+ 113us) client_negotiation.cc:769] Sending connection context
0811 05:13:46.527222 (+ 141us) client_negotiation.cc:240] Negotiation successful
0811 05:13:46.527384 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 05:13:46.528868 17360 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.7.67.188:36501: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb"
member_type: NON_VOTER
last_known_addr {
host: "127.7.67.188"
port: 36501
}
attrs {
promote: true
}
}
dest_uuid: "95667e6974cb44ba9111427a619ddcef"
cas_config_opid_index: 5
I20260811 05:13:46.529492 17360 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: NON_VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: true } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: NON_VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: true } }
I20260811 05:13:46.530676 18231 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6f60f0e75bad4042a07087df578dfbdb (127.7.67.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: NON_VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: true } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: NON_VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: true } } }
I20260811 05:13:46.532390 18231 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.7.67.188:36501
I20260811 05:13:46.532648 18231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "95667e6974cb44ba9111427a619ddcef" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: NON_VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: true } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: NON_VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: true } } }
I20260811 05:13:46.532768 18231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:46.533282 18429 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:46.534801 18429 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 1 FOLLOWER]: Refusing update from remote peer 95667e6974cb44ba9111427a619ddcef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 05:13:46.536617 18237 master.cc:537] Master@127.7.67.188:36501 shutting down...
I20260811 05:13:46.536481 18232 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: NON_VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 05:13:46.539031 17343 consensus_peers.cc:487] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef -> Peer 6f60f0e75bad4042a07087df578dfbdb (127.7.67.188:36501): Couldn't send request to peer 6f60f0e75bad4042a07087df578dfbdb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 05:13:46.539418 18229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.530883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:40365 (local address 127.0.0.1:44292)
0811 05:13:46.531646 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:46.531661 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:46.531778 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:46.532264 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:46.532267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:46.532288 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:46.532377 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.532386 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.533566 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.533569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.534982 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.534994 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.538889 (+ 3895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.538896 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.539013 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.539069 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 05:13:46.539155 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 05:13:46.539264 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693}
I20260811 05:13:46.540007 18469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.531135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44292 (local address 127.7.67.189:40365)
0811 05:13:46.531554 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:46.531559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:46.531714 (+ 155us) server_negotiation.cc:407] Connection header received
0811 05:13:46.531900 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:46.531907 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:46.531976 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:46.532056 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:46.532580 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.533396 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.535145 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.535930 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.536021 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.536204 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:46.539633 (+ 3429us) server_negotiation.cc:299] Negotiation successful
0811 05:13:46.539845 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335}
I20260811 05:13:46.540504 18304 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 1 LEARNER]: Refusing update from remote peer 95667e6974cb44ba9111427a619ddcef: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 05:13:46.541060 18232 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: NON_VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 05:13:46.543042 18304 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 6f60f0e75bad4042a07087df578dfbdb (127.7.67.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: NON_VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: true } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: NON_VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: true } } }
I20260811 05:13:46.545991 18344 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95667e6974cb44ba9111427a619ddcef. Latest consensus state: current_term: 1 leader_uuid: "95667e6974cb44ba9111427a619ddcef" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: NON_VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: true } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: NON_VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: true } } }
I20260811 05:13:46.546118 18344 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [sys.catalog]: This master's current role is: LEARNER
I20260811 05:13:46.546216 18344 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "95667e6974cb44ba9111427a619ddcef" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: NON_VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: true } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: NON_VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: true } } }
I20260811 05:13:46.546300 18344 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [sys.catalog]: This master's current role is: LEARNER
I20260811 05:13:46.546875 18232 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef: attempting to promote NON_VOTER ba069f98f23c4f3b9125de6627e8992d to VOTER
I20260811 05:13:46.547324 18232 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: NON_VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: true } }
I20260811 05:13:46.549255 18304 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 1 LEARNER]: Refusing update from remote peer 95667e6974cb44ba9111427a619ddcef: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260811 05:13:46.549336 17343 consensus_peers.cc:487] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef -> Peer 6f60f0e75bad4042a07087df578dfbdb (127.7.67.188:36501): Couldn't send request to peer 6f60f0e75bad4042a07087df578dfbdb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 05:13:46.549674 18231 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 05:13:46.550355 18237 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:46.550657 18237 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:46.550786 18237 tablet_replica.cc:330] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb: stopping tablet replica
I20260811 05:13:46.552050 18231 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ba069f98f23c4f3b9125de6627e8992d (127.7.67.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: NON_VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: true } } }
I20260811 05:13:46.554301 18304 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, ba069f98f23c4f3b9125de6627e8992d (127.7.67.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: NON_VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: true } } }
I20260811 05:13:46.555930 18504 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "95667e6974cb44ba9111427a619ddcef" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: NON_VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: true } } }
I20260811 05:13:46.556699 18231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "95667e6974cb44ba9111427a619ddcef" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: NON_VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: true } } }
I20260811 05:13:46.556824 18231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:46.556957 18231 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "95667e6974cb44ba9111427a619ddcef" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: NON_VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: true } } }
I20260811 05:13:46.557058 18231 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:46.558034 18237 master.cc:552] Master@127.7.67.188:36501 shutdown complete.
I20260811 05:13:46.558111 18237 master_runner.cc:305] Clearing existing system tablet
I20260811 05:13:46.558360 18237 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:46.558437 18237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:46.559561 18504 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:13:46.559974 18237 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:46.560961 18507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:46.561172 18237 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:46.561318 18237 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:46.561364 18237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/wal
uuid: "6f60f0e75bad4042a07087df578dfbdb"
format_stamp: "Formatted at 2026-08-11 05:13:46 on dist-test-slave-04bb"
server_key: "f146197f431ad270f46faa3fc0b22e2b"
server_key_iv: "239b7119192aa69a5de28b45a29dc2d9"
server_key_version: "encryptionkey@0"
I20260811 05:13:46.563200 18237 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:46.565516 18237 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 05:13:46.565630 18237 log.cc:1196] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 05:13:46.565902 18237 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb: Deleting consensus metadata
I20260811 05:13:46.566097 18237 master_runner.cc:315] Copying system tablet from 127.7.67.190:45385
I20260811 05:13:46.567325 18237 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb: tablet copy: Beginning tablet copy session from remote peer at address 127.7.67.190:45385
I20260811 05:13:46.576398 17429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.567956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57366 (local address 127.7.67.190:45385)
0811 05:13:46.568089 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:46.568094 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:46.568322 (+ 228us) server_negotiation.cc:407] Connection header received
0811 05:13:46.568485 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:46.568489 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:46.568565 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:46.568666 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:46.569128 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.569713 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.570760 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.571007 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.571159 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.572759 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:46.572764 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:46.572767 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:46.572794 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:46.574526 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.575019 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.575023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.575025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.575166 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.575473 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.575478 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.575480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.575676 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:46.575797 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:46.576094 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 05:13:46.576223 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:13:46.576411 18518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.567909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.0.0.1:57366)
0811 05:13:46.568222 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:46.568235 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:46.568373 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:46.568683 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:46.568686 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:46.568914 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:46.569002 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.569008 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.569835 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.569840 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.570609 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.570620 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.571277 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.571280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.571349 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.571947 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:46.571960 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:46.572509 (+ 549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:46.574663 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.574667 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.574669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.574899 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.575268 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.575272 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.575275 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.575368 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.575787 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:46.575790 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:46.575905 (+ 115us) client_negotiation.cc:769] Sending connection context
0811 05:13:46.576033 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 05:13:46.576191 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":91,"threads_started":1}
I20260811 05:13:46.577232 17391 tablet_copy_service.cc:140] P 95667e6974cb44ba9111427a619ddcef: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 6f60f0e75bad4042a07087df578dfbdb ({username='oryx', principal='oryx/127.7.67.188@KRBTEST.COM'} at 127.0.0.1:57366)
I20260811 05:13:46.577344 17391 tablet_copy_service.cc:161] P 95667e6974cb44ba9111427a619ddcef: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 6f60f0e75bad4042a07087df578dfbdb at {username='oryx', principal='oryx/127.7.67.188@KRBTEST.COM'} at 127.0.0.1:57366: session id = 6f60f0e75bad4042a07087df578dfbdb-00000000000000000000000000000000
I20260811 05:13:46.578226 17391 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 05:13:46.579454 18237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 05:13:46.581574 18237 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb: tablet copy: Starting download of 0 data blocks...
I20260811 05:13:46.581810 18237 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb: tablet copy: Starting download of 1 WAL segments...
I20260811 05:13:46.583482 18237 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 05:13:46.585161 17391 tablet_copy_service.cc:342] P 95667e6974cb44ba9111427a619ddcef: Request end of tablet copy session 6f60f0e75bad4042a07087df578dfbdb-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.7.67.188@KRBTEST.COM'} at 127.0.0.1:57366
I20260811 05:13:46.585283 17391 tablet_copy_service.cc:433] P 95667e6974cb44ba9111427a619ddcef: ending tablet copy session 6f60f0e75bad4042a07087df578dfbdb-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 6f60f0e75bad4042a07087df578dfbdb
I20260811 05:13:46.592468 18237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:46.594102 18519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:46.594887 18523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:46.595360 18237 server_base.cc:1034] running on GCE node
W20260811 05:13:46.596706 18520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:46.596933 18237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:46.597347 18237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:46.598498 18237 hybrid_clock.cc:648] HybridClock initialized: now 1786425226598508 us; error 46 us; skew 500 ppm
Aug 11 05:13:46 dist-test-slave-04bb krb5kdc[17292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425226, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:46.600214 18237 init.cc:377] Logged in from keytab as oryx/127.7.67.188@KRBTEST.COM (short username oryx)
I20260811 05:13:46.600842 18237 webserver.cc:460] Webserver started at http://127.7.67.188:44831/ using document root <none> and password file <none>
I20260811 05:13:46.601042 18237 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:46.601080 18237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:46.609241 18237 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.001s sys 0.000s
I20260811 05:13:46.610265 18538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:46.610540 18237 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:46.610646 18237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/wal
uuid: "6f60f0e75bad4042a07087df578dfbdb"
format_stamp: "Formatted at 2026-08-11 05:13:46 on dist-test-slave-04bb"
server_key: "f146197f431ad270f46faa3fc0b22e2b"
server_key_iv: "239b7119192aa69a5de28b45a29dc2d9"
server_key_version: "encryptionkey@0"
I20260811 05:13:46.610760 18237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:46.625340 17865 heartbeater.cc:503] Master 127.7.67.190:45385 was elected leader, sending a full tablet report...
I20260811 05:13:46.634547 18237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:46.637048 18237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:46.642706 18237 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.188:36501
I20260811 05:13:46.642740 18612 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.188:36501 every 8 connection(s)
I20260811 05:13:46.643306 18237 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/master-2/data/info.pb
I20260811 05:13:46.644956 18613 sys_catalog.cc:263] Verifying existing consensus state
I20260811 05:13:46.645861 18613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb: Bootstrap starting.
I20260811 05:13:46.657234 18613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:13:46.657547 18613 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb: Bootstrap complete.
I20260811 05:13:46.658097 18613 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: NON_VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: true } }
I20260811 05:13:46.658241 18613 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 6f60f0e75bad4042a07087df578dfbdb, State: Initialized, Role: LEARNER
I20260811 05:13:46.658381 18613 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: NON_VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: true } }
I20260811 05:13:46.658800 18620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: NON_VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: true } } }
I20260811 05:13:46.658978 18613 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:46.658983 18620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [sys.catalog]: This master's current role is: LEARNER
I20260811 05:13:46.663594 18637 catalog_manager.cc:1259] Loaded cluster ID: e9f81071b52b4f468d783cb68b56dd85
I20260811 05:13:46.663676 18637 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb: loading cluster ID for follower catalog manager: success
I20260811 05:13:46.665366 18637 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb: acquiring CA information for follower catalog manager: success
I20260811 05:13:46.666494 18637 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 05:13:46.679376 18500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:46.679665 18500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:46.679723 18500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:46.685664 18500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:46.685726 18500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:46.685806 18500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:13:46.692451 18500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:45919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.129.keytab
--principal=oryx/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:36481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=46679
--tserver_master_addrs=127.7.67.190:45385,127.7.67.189:40365,127.7.67.188:36501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18500
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:46.693421 18500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:46.694532 18500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:46.704438 18649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:46.704430 18645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:46.704432 18646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:46.704584 18500 server_base.cc:1034] running on GCE node
I20260811 05:13:46.705091 18500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:46.705734 18500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:46.706928 18500 hybrid_clock.cc:648] HybridClock initialized: now 1786425226706916 us; error 28 us; skew 500 ppm
Aug 11 05:13:46 dist-test-slave-04bb krb5kdc[17292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425226, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:46.709758 18500 init.cc:377] Logged in from keytab as oryx/127.7.67.129@KRBTEST.COM (short username oryx)
I20260811 05:13:46.710675 18500 webserver.cc:460] Webserver started at http://127.7.67.129:46679/ using document root <none> and password file <none>
I20260811 05:13:46.711293 18500 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:46.711414 18500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:46.715291 18500 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260811 05:13:46.717501 18666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:46.718792 18500 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:13:46.718947 18500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "d2e72a180c5e43a5ba0728584cc25be4"
format_stamp: "Formatted at 2026-08-11 05:13:45 on dist-test-slave-04bb"
server_key: "b89a5a680b302a9df49114421abd7463"
server_key_iv: "9f0cf358dbc9da2d70a9743d097976ba"
server_key_version: "encryptionkey@0"
I20260811 05:13:46.719493 18500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:46.742460 18500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:46.745414 18500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:46.745623 18500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:46.746201 18500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:46.747123 18500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:46.747195 18500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:46.747262 18500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:46.747293 18500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:46.758097 18500 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:36481
I20260811 05:13:46.758116 18850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:36481 every 8 connection(s)
I20260811 05:13:46.759152 18500 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:46.769297 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 18500
I20260811 05:13:46.769515 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 17677
Aug 11 05:13:46 dist-test-slave-04bb krb5kdc[17292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425226, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM
Aug 11 05:13:46 dist-test-slave-04bb krb5kdc[17292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425226, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.189@KRBTEST.COM
Aug 11 05:13:46 dist-test-slave-04bb krb5kdc[17292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425226, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.188@KRBTEST.COM
I20260811 05:13:46.795562 17429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.763011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:52071 (local address 127.7.67.190:45385)
0811 05:13:46.763162 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:46.763167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:46.763356 (+ 189us) server_negotiation.cc:407] Connection header received
0811 05:13:46.763943 (+ 587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:46.763947 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:46.763996 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:46.764078 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:46.765390 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.766196 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.767611 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.767899 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.767987 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.789877 (+ 21890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:46.789882 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:46.789885 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:46.789928 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:46.792682 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.793474 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.793478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.793480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.793572 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.794037 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.794042 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.794047 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.794328 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:46.794406 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:46.795245 (+ 839us) server_negotiation.cc:299] Negotiation successful
0811 05:13:46.795388 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 05:13:46.795656 18469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.761435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:52049 (local address 127.7.67.189:40365)
0811 05:13:46.761673 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:46.761678 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:46.762451 (+ 773us) server_negotiation.cc:407] Connection header received
0811 05:13:46.763755 (+ 1304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:46.763764 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:46.763815 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:46.763909 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:46.765390 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.766197 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.767632 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.767913 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.768049 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.791089 (+ 23040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:46.791096 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:46.791102 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:46.791139 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:46.792971 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.793582 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.793588 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.793593 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.793687 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.794121 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.794128 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.794133 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.794429 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:46.794529 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:46.795272 (+ 743us) server_negotiation.cc:299] Negotiation successful
0811 05:13:46.795418 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260811 05:13:46.796252 18862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.761474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:35517 (local address 127.7.67.188:36501)
0811 05:13:46.761743 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:46.761755 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:46.762481 (+ 726us) server_negotiation.cc:407] Connection header received
0811 05:13:46.763636 (+ 1155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:46.763639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:46.763713 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:46.763803 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:46.765368 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.766072 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.767238 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.767507 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.767625 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.790877 (+ 23252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:46.790886 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:46.790890 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:46.790929 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:46.793902 (+ 2973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.794673 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.794676 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.794678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.794766 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:46.795066 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:46.795069 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:46.795070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:46.795309 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:46.795401 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:46.795959 (+ 558us) server_negotiation.cc:299] Negotiation successful
0811 05:13:46.796085 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":85,"threads_started":1}
I20260811 05:13:46.796269 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:40403
--local_ip_for_outbound_sockets=127.7.67.130
--tserver_master_addrs=127.7.67.190:45385
--webserver_port=41043
--webserver_interface=127.7.67.130
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.130.keytab
--principal=oryx/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:45919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.67.190:45385,127.7.67.189:40365
--tserver_master_addrs=127.7.67.190:45385,127.7.67.189:40365,127.7.67.188:36501 with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:13:46.797309 18864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.762049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:36501 (local address 127.7.67.129:35517)
0811 05:13:46.762382 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:46.762397 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:46.763449 (+ 1052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:46.763977 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:46.763987 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:46.764516 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:46.765222 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.765235 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.766221 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.766224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.767105 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.767115 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.768095 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.768097 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.768324 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.769249 (+ 925us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:46.769266 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:46.790654 (+ 21388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:46.794246 (+ 3592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.794249 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.794251 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.794561 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.794874 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.794876 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.794878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.794958 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.795461 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:46.795464 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:46.795546 (+ 82us) client_negotiation.cc:769] Sending connection context
0811 05:13:46.795667 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 05:13:46.796693 (+ 1026us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":942,"spinlock_wait_cycles":10240,"thread_start_us":190,"threads_started":1}
I20260811 05:13:46.797314 18863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.761813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:40365 (local address 127.7.67.129:52049)
0811 05:13:46.762299 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:46.762350 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:46.763432 (+ 1082us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:46.763980 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:46.763989 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:46.764562 (+ 573us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:46.765233 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.765243 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.766352 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.766355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.767483 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.767493 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.768193 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.768198 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.768368 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.769163 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:46.769200 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:46.790889 (+ 21689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:46.793111 (+ 2222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.793117 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.793122 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.793458 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.793810 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.793816 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.793821 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.793946 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.794633 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:46.794638 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:46.794936 (+ 298us) client_negotiation.cc:769] Sending connection context
0811 05:13:46.796723 (+ 1787us) client_negotiation.cc:240] Negotiation successful
0811 05:13:46.796908 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":373,"spinlock_wait_cycles":96512,"thread_start_us":133,"threads_started":1}
I20260811 05:13:46.798063 18865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.762840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.129:52071)
0811 05:13:46.763266 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:46.763278 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:46.763449 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:46.764115 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:46.764119 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:46.764562 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:46.765233 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.765248 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.766352 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.766355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.767483 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.767493 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.768065 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.768069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.768325 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.769163 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:46.769190 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:46.789567 (+ 20377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:46.792835 (+ 3268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.792842 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.792891 (+ 49us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.793333 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.793719 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:46.793723 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:46.793725 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:46.793920 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:46.794496 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:46.794503 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:46.794932 (+ 429us) client_negotiation.cc:769] Sending connection context
0811 05:13:46.795806 (+ 874us) client_negotiation.cc:240] Negotiation successful
0811 05:13:46.796051 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":340,"spinlock_wait_cycles":40960,"thread_start_us":201,"threads_started":1}
I20260811 05:13:46.798805 18852 heartbeater.cc:347] Connected to a master server at 127.7.67.188:36501
I20260811 05:13:46.798837 18854 heartbeater.cc:347] Connected to a master server at 127.7.67.189:40365
I20260811 05:13:46.798873 18851 heartbeater.cc:347] Connected to a master server at 127.7.67.190:45385
I20260811 05:13:46.799067 18852 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:46.799068 18854 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:46.799067 18851 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:46.799712 18852 heartbeater.cc:511] Master 127.7.67.188:36501 requested a full tablet report, sending...
I20260811 05:13:46.799780 18854 heartbeater.cc:511] Master 127.7.67.189:40365 requested a full tablet report, sending...
I20260811 05:13:46.800040 18851 heartbeater.cc:511] Master 127.7.67.190:45385 requested a full tablet report, sending...
I20260811 05:13:46.801186 17360 ts_manager.cc:194] Re-registered known tserver with Master: d2e72a180c5e43a5ba0728584cc25be4 (127.7.67.129:36481)
I20260811 05:13:46.801517 18288 ts_manager.cc:194] Registered new tserver with Master: d2e72a180c5e43a5ba0728584cc25be4 (127.7.67.129:36481)
I20260811 05:13:46.801858 17360 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.67.129@KRBTEST.COM'} at 127.7.67.129:52071
I20260811 05:13:46.802645 18564 ts_manager.cc:194] Registered new tserver with Master: d2e72a180c5e43a5ba0728584cc25be4 (127.7.67.129:36481)
I20260811 05:13:46.841161 18065 heartbeater.cc:503] Master 127.7.67.190:45385 was elected leader, sending a full tablet report...
W20260811 05:13:46.912611 18866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:46.912889 18866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:46.912957 18866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:46.916891 18866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:46.916962 18866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:46.917047 18866 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:13:46.922276 18866 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:45919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.130.keytab
--principal=oryx/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:40403
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=41043
--tserver_master_addrs=127.7.67.190:45385,127.7.67.189:40365,127.7.67.188:36501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18866
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:46.923374 18866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:46.924554 18866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:46.934377 18874 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:46.934482 18866 server_base.cc:1034] running on GCE node
W20260811 05:13:46.934386 18873 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:46.934377 18877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:46.935052 18866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:46.935664 18866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:46.936831 18866 hybrid_clock.cc:648] HybridClock initialized: now 1786425226936801 us; error 42 us; skew 500 ppm
Aug 11 05:13:46 dist-test-slave-04bb krb5kdc[17292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425226, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:46.939841 18866 init.cc:377] Logged in from keytab as oryx/127.7.67.130@KRBTEST.COM (short username oryx)
I20260811 05:13:46.940938 18866 webserver.cc:460] Webserver started at http://127.7.67.130:41043/ using document root <none> and password file <none>
I20260811 05:13:46.941540 18866 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:46.941641 18866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:46.945468 18866 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 05:13:46.947511 18886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:46.948834 18866 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:13:46.948954 18866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "82b986efd42f463ba147b7067465ac5e"
format_stamp: "Formatted at 2026-08-11 05:13:45 on dist-test-slave-04bb"
server_key: "c9af80dea12b953e678c67f86c8b7c6c"
server_key_iv: "a6f852ed55e8c41cbd3cef15cc77f489"
server_key_version: "encryptionkey@0"
I20260811 05:13:46.949471 18866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:46.968405 18866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:46.971362 18866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:46.971563 18866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:46.972186 18866 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:46.973089 18866 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:46.973135 18866 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:46.973207 18866 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:46.973241 18866 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:46.982226 18866 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:40403
I20260811 05:13:46.982264 19066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:40403 every 8 connection(s)
I20260811 05:13:46.983209 18866 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:46.984730 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 18866
I20260811 05:13:46.984925 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 17874
Aug 11 05:13:46 dist-test-slave-04bb krb5kdc[17292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425226, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.188@KRBTEST.COM
I20260811 05:13:47.005152 18862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.990971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:48793 (local address 127.7.67.188:36501)
0811 05:13:46.991136 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:46.991144 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:46.991163 (+ 19us) server_negotiation.cc:407] Connection header received
0811 05:13:46.991221 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:46.991227 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:46.991291 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:46.991383 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:46.993234 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.994059 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.995983 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.996397 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.996493 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.000082 (+ 3589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:47.000090 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:47.000096 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:47.000145 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:47.002196 (+ 2051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.002894 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.002902 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.002907 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.003005 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.003460 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.003466 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.003471 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.003736 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:47.003843 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:47.004660 (+ 817us) server_negotiation.cc:299] Negotiation successful
0811 05:13:47.004868 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 05:13:47.006031 19072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.985630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:36501 (local address 127.7.67.130:48793)
0811 05:13:46.988069 (+ 2439us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:46.988149 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:46.989648 (+ 1499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:46.991720 (+ 2072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:46.991733 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:46.992299 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:46.993014 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.993037 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.994257 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.994263 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.995808 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:46.995825 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:46.996763 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:46.996770 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:46.997010 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:46.997779 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:46.997808 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:46.999866 (+ 2058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:47.002363 (+ 2497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:47.002372 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:47.002391 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:47.002767 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:47.003140 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:47.003146 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:47.003151 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:47.003340 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:47.003923 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:47.003933 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:47.004272 (+ 339us) client_negotiation.cc:769] Sending connection context
0811 05:13:47.004548 (+ 276us) client_negotiation.cc:240] Negotiation successful
0811 05:13:47.004927 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2164,"thread_start_us":1928,"threads_started":1}
I20260811 05:13:47.011162 19068 heartbeater.cc:347] Connected to a master server at 127.7.67.188:36501
I20260811 05:13:47.011380 19068 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:47.012048 19068 heartbeater.cc:511] Master 127.7.67.188:36501 requested a full tablet report, sending...
I20260811 05:13:47.013487 18564 ts_manager.cc:194] Registered new tserver with Master: 82b986efd42f463ba147b7067465ac5e (127.7.67.130:40403)
Aug 11 05:13:47 dist-test-slave-04bb krb5kdc[17292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425226, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.189@KRBTEST.COM
Aug 11 05:13:47 dist-test-slave-04bb krb5kdc[17292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425226, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM
I20260811 05:13:47.017679 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:43549
--local_ip_for_outbound_sockets=127.7.67.131
--tserver_master_addrs=127.7.67.190:45385
--webserver_port=34601
--webserver_interface=127.7.67.131
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.131.keytab
--principal=oryx/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:45919
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.7.67.190:45385,127.7.67.189:40365
--tserver_master_addrs=127.7.67.190:45385,127.7.67.189:40365,127.7.67.188:36501 with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:13:47.020687 19072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.985632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:40365 (local address 127.7.67.130:33295)
0811 05:13:47.009768 (+ 24136us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:47.009784 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:47.009919 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:47.010424 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:47.010429 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:47.010639 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:47.010779 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:47.010791 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.012229 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.012233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:47.013201 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:47.013213 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.014099 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.014103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:47.014229 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.014929 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:47.014953 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:47.016761 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:47.018777 (+ 2016us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:47.018783 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:47.018788 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:47.019057 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:47.019420 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:47.019424 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:47.019426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:47.019561 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:47.020125 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:47.020127 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:47.020239 (+ 112us) client_negotiation.cc:769] Sending connection context
0811 05:13:47.020343 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 05:13:47.020482 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24027,"mutex_wait_us":23835}
I20260811 05:13:47.020853 18469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:46.987863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:33295 (local address 127.7.67.189:40365)
0811 05:13:46.988032 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:46.988037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:47.009882 (+ 21845us) server_negotiation.cc:407] Connection header received
0811 05:13:47.010071 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:47.010077 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:47.010165 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:47.010240 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:47.010960 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.011890 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.013364 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.013754 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.013861 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.016907 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:47.016911 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:47.016914 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:47.016948 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:47.018649 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.019193 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.019198 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.019200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.019295 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.019679 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.019686 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.019689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.019999 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:47.020105 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:47.020447 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 05:13:47.020632 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 05:13:47.021543 19076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.009771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.130:51389)
0811 05:13:47.010120 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:47.010138 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:47.010310 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:47.011010 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:47.011013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:47.011251 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:47.011395 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:47.011404 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.012708 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.012712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:47.013719 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:47.013730 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.014549 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.014554 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:47.014719 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.015234 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:47.015254 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:47.017444 (+ 2190us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:47.019684 (+ 2240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:47.019687 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:47.019689 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:47.019987 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:47.020322 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:47.020327 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:47.020331 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:47.020414 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:47.021035 (+ 621us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:47.021039 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:47.021127 (+ 88us) client_negotiation.cc:769] Sending connection context
0811 05:13:47.021236 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 05:13:47.021370 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":100,"threads_started":1}
I20260811 05:13:47.021548 17429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.009213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:51389 (local address 127.7.67.190:45385)
0811 05:13:47.009392 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:47.009400 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:47.010231 (+ 831us) server_negotiation.cc:407] Connection header received
0811 05:13:47.010516 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:47.010521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:47.010588 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:47.010695 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:47.011656 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.012474 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.013882 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.014247 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.014346 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.017589 (+ 3243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:47.017592 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:47.017595 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:47.017625 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:47.019549 (+ 1924us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.020104 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.020109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.020112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.020210 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.020569 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.020572 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.020574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.020816 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:47.020901 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:47.021293 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 05:13:47.021409 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 05:13:47.021641 19069 heartbeater.cc:347] Connected to a master server at 127.7.67.189:40365
I20260811 05:13:47.021757 19069 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:47.021970 19069 heartbeater.cc:511] Master 127.7.67.189:40365 requested a full tablet report, sending...
I20260811 05:13:47.022109 19067 heartbeater.cc:347] Connected to a master server at 127.7.67.190:45385
I20260811 05:13:47.022192 19067 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:47.022383 19067 heartbeater.cc:511] Master 127.7.67.190:45385 requested a full tablet report, sending...
I20260811 05:13:47.022900 18288 ts_manager.cc:194] Registered new tserver with Master: 82b986efd42f463ba147b7067465ac5e (127.7.67.130:40403)
I20260811 05:13:47.023149 17360 ts_manager.cc:194] Re-registered known tserver with Master: 82b986efd42f463ba147b7067465ac5e (127.7.67.130:40403)
I20260811 05:13:47.024127 17360 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.67.130@KRBTEST.COM'} at 127.7.67.130:51389
I20260811 05:13:47.063771 18862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.059907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55780 (local address 127.7.67.188:36501)
0811 05:13:47.060073 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:47.060077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:47.060357 (+ 280us) server_negotiation.cc:407] Connection header received
0811 05:13:47.060571 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:47.060574 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:47.060641 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:47.060749 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:47.061090 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.061679 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.062752 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.063184 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.063311 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.063470 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:47.063553 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 05:13:47.063623 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 05:13:47.063772 19079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.059834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:36501 (local address 127.0.0.1:55780)
0811 05:13:47.060243 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:47.060256 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:47.060420 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:47.060777 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:47.060780 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:47.060822 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:47.060948 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:47.060954 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.061795 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.061799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:47.062613 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:47.062622 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.063331 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.063335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:47.063425 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.063449 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 05:13:47.063540 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 05:13:47.063624 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":76,"threads_started":1}
I20260811 05:13:47.065040 18579 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 05:13:47.065423 18620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95667e6974cb44ba9111427a619ddcef. Latest consensus state: current_term: 1 leader_uuid: "95667e6974cb44ba9111427a619ddcef" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: NON_VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: true } } }
I20260811 05:13:47.065600 18620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [sys.catalog]: This master's current role is: LEARNER
W20260811 05:13:47.136952 19078 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:47.137239 19078 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:47.137281 19078 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:47.141310 19078 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:47.141362 19078 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:47.141418 19078 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:13:47.146507 19078 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:45919
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.131.keytab
--principal=oryx/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:43549
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=34601
--tserver_master_addrs=127.7.67.190:45385,127.7.67.189:40365,127.7.67.188:36501
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19078
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:47.147473 19078 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:47.148445 19078 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:47.158244 19086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:47.158259 19085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:47.158289 19090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:47.158509 19078 server_base.cc:1034] running on GCE node
I20260811 05:13:47.159039 19078 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:47.159701 19078 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:47.160914 19078 hybrid_clock.cc:648] HybridClock initialized: now 1786425227160892 us; error 45 us; skew 500 ppm
Aug 11 05:13:47 dist-test-slave-04bb krb5kdc[17292](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425227, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:47.163929 19078 init.cc:377] Logged in from keytab as oryx/127.7.67.131@KRBTEST.COM (short username oryx)
I20260811 05:13:47.164893 19078 webserver.cc:460] Webserver started at http://127.7.67.131:34601/ using document root <none> and password file <none>
I20260811 05:13:47.165475 19078 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:47.165591 19078 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:47.169567 19078 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:13:47.171670 19102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:47.173008 19078 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:13:47.173153 19078 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "32e63505878c446594d70821360ed6c5"
format_stamp: "Formatted at 2026-08-11 05:13:45 on dist-test-slave-04bb"
server_key: "9ef99922e2724690e2dae84ea665c2e4"
server_key_iv: "b3d14fcca1d71a56ba54c1684604f392"
server_key_version: "encryptionkey@0"
I20260811 05:13:47.173699 19078 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:47.188786 19078 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:47.191751 19078 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:47.191903 19078 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:47.192425 19078 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:47.193323 19078 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:47.193372 19078 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:47.193410 19078 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:47.193426 19078 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:47.202502 19078 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:43549
I20260811 05:13:47.202577 19274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:43549 every 8 connection(s)
I20260811 05:13:47.203559 19078 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:13:47.207300 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 19078
Aug 11 05:13:47 dist-test-slave-04bb krb5kdc[17292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425227, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM
Aug 11 05:13:47 dist-test-slave-04bb krb5kdc[17292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425227, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.188@KRBTEST.COM
Aug 11 05:13:47 dist-test-slave-04bb krb5kdc[17292](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425227, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.189@KRBTEST.COM
I20260811 05:13:47.221369 19292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.209454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44302 (local address 127.7.67.189:40365)
0811 05:13:47.210044 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:47.210048 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:47.210376 (+ 328us) server_negotiation.cc:407] Connection header received
0811 05:13:47.210415 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:47.210418 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:47.210462 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:47.210532 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:47.211662 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.212489 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.213613 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.213963 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.214078 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.216429 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:47.216433 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:47.216436 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:47.216464 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:47.218902 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.219449 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.219452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.219454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.219546 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.219846 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.219850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.219852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.220063 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:47.220184 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:47.221026 (+ 842us) server_negotiation.cc:299] Negotiation successful
0811 05:13:47.221158 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":88,"threads_started":1}
I20260811 05:13:47.224419 18862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.205471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:53387 (local address 127.7.67.188:36501)
0811 05:13:47.205596 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:47.205600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:47.206539 (+ 939us) server_negotiation.cc:407] Connection header received
0811 05:13:47.207771 (+ 1232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:47.207775 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:47.207844 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:47.207945 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:47.209902 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.210720 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.212687 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.213023 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.213101 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.218784 (+ 5683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:47.218789 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:47.218794 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:47.218832 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:47.220670 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.221323 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.221326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.221328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.221414 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.222137 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.222141 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.222144 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.222378 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:47.222483 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:47.224130 (+ 1647us) server_negotiation.cc:299] Negotiation successful
0811 05:13:47.224263 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 05:13:47.225485 19281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.205878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:36501 (local address 127.7.67.131:53387)
0811 05:13:47.206376 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:47.206428 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:47.207557 (+ 1129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:47.208256 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:47.208265 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:47.209477 (+ 1212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:47.209728 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:47.209750 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.210882 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.210885 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:47.212545 (+ 1660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:47.212558 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.213307 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.213311 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:47.214176 (+ 865us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.214655 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:47.214672 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:47.218614 (+ 3942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:47.220815 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:47.220821 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:47.220839 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:47.221205 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:47.221512 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:47.221515 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:47.221517 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:47.222009 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:47.222543 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:47.222552 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:47.222907 (+ 355us) client_negotiation.cc:769] Sending connection context
0811 05:13:47.223131 (+ 224us) client_negotiation.cc:240] Negotiation successful
0811 05:13:47.223505 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":905,"spinlock_wait_cycles":15744,"thread_start_us":138,"threads_started":1}
I20260811 05:13:47.225849 19282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.206921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.131:57563)
0811 05:13:47.208000 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:47.208015 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:47.208148 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:47.208558 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:47.208561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:47.208936 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:47.209730 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:47.209749 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.210859 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.210864 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:47.212678 (+ 1814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:47.212683 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.213358 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.213362 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:47.213514 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.214385 (+ 871us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:47.214403 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:47.216597 (+ 2194us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:47.222158 (+ 5561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:47.222161 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:47.222163 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:47.222425 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:47.224021 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:47.224025 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:47.224030 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:47.224130 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:47.224757 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:47.224760 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:47.224854 (+ 94us) client_negotiation.cc:769] Sending connection context
0811 05:13:47.224942 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 05:13:47.225072 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1018,"mutex_wait_us":440,"spinlock_wait_cycles":17536,"thread_start_us":79,"threads_started":1}
I20260811 05:13:47.226660 17429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.207104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:57563 (local address 127.7.67.190:45385)
0811 05:13:47.207248 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:47.207253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:47.208084 (+ 831us) server_negotiation.cc:407] Connection header received
0811 05:13:47.208264 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:47.208268 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:47.208317 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:47.208372 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:47.209915 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.210684 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.212792 (+ 2108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.213127 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.213224 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.216823 (+ 3599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:47.216827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:47.216829 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:47.216859 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:47.222045 (+ 5186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.223494 (+ 1449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.223498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.223500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.223605 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.224254 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.224258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.224259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.224481 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:47.224720 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:47.226397 (+ 1677us) server_negotiation.cc:299] Negotiation successful
0811 05:13:47.226518 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 05:13:47.226830 19277 heartbeater.cc:347] Connected to a master server at 127.7.67.188:36501
I20260811 05:13:47.227095 19277 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:47.227713 19277 heartbeater.cc:511] Master 127.7.67.188:36501 requested a full tablet report, sending...
I20260811 05:13:47.227762 19294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.212677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57368 (local address 127.7.67.190:45385)
0811 05:13:47.217232 (+ 4555us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:47.217237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:47.217252 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:13:47.217296 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:47.217301 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:47.217358 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:47.217452 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:47.218422 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.219322 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.220566 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.221054 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.221149 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.223728 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:47.223734 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:47.223739 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:47.223785 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:47.225714 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.226251 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.226255 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.226256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.226357 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.226699 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.226703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.226704 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.226969 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:47.227088 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:47.227469 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 05:13:47.227596 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4464,"thread_start_us":77,"threads_started":1}
I20260811 05:13:47.229053 18564 ts_manager.cc:194] Registered new tserver with Master: 32e63505878c446594d70821360ed6c5 (127.7.67.131:43549)
I20260811 05:13:47.229552 19275 heartbeater.cc:347] Connected to a master server at 127.7.67.190:45385
I20260811 05:13:47.229635 19275 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:47.229804 19275 heartbeater.cc:511] Master 127.7.67.190:45385 requested a full tablet report, sending...
I20260811 05:13:47.229951 19297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.215349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55792 (local address 127.7.67.188:36501)
0811 05:13:47.215608 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:47.215613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:47.219574 (+ 3961us) server_negotiation.cc:407] Connection header received
0811 05:13:47.219621 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:47.219626 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:47.219689 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:47.219775 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:47.220973 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.221736 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.222787 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.223267 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.223381 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.225382 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:47.225386 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:47.225391 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:47.225421 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:47.227813 (+ 2392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.228285 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.228289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.228291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.228389 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.228738 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.228741 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.228743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.228954 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:47.229073 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:47.229668 (+ 595us) server_negotiation.cc:299] Negotiation successful
0811 05:13:47.229801 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":116,"threads_started":1}
I20260811 05:13:47.231117 17360 ts_manager.cc:194] Re-registered known tserver with Master: 32e63505878c446594d70821360ed6c5 (127.7.67.131:43549)
I20260811 05:13:47.231180 19280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.205878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:40365 (local address 127.7.67.131:51403)
0811 05:13:47.206376 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:47.206428 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:47.207570 (+ 1142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:47.208260 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:47.208267 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:47.209101 (+ 834us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:47.209726 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:47.209748 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.210892 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.210894 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:47.212547 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:47.212560 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.213390 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.213393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:47.213561 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.214317 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:47.214343 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:47.217652 (+ 3309us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:47.223370 (+ 5718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:47.223373 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:47.223375 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:47.223927 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:47.230076 (+ 6149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:47.230081 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:47.230084 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:47.230185 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:47.230639 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:47.230643 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:47.230751 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 05:13:47.230859 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 05:13:47.231004 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":286,"spinlock_wait_cycles":13440,"thread_start_us":132,"threads_started":1}
I20260811 05:13:47.231163 18469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.205445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:51403 (local address 127.7.67.189:40365)
0811 05:13:47.205590 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:47.205596 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:47.206543 (+ 947us) server_negotiation.cc:407] Connection header received
0811 05:13:47.207816 (+ 1273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:47.207820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:47.207893 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:47.207954 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:47.209901 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.210737 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.212842 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.213160 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.213267 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.217810 (+ 4543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:47.217815 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:47.217819 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:47.217846 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:47.223216 (+ 5370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.228719 (+ 5503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.228723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.228724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.228827 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.230303 (+ 1476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.230306 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.230308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.230520 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:47.230630 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:47.230917 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 05:13:47.231029 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 05:13:47.231803 17360 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.67.131@KRBTEST.COM'} at 127.7.67.131:57563
I20260811 05:13:47.232061 17361 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57368:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:13:47.235145 17361 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:47.237327 19278 heartbeater.cc:347] Connected to a master server at 127.7.67.189:40365
I20260811 05:13:47.237434 19278 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:47.237645 19278 heartbeater.cc:511] Master 127.7.67.189:40365 requested a full tablet report, sending...
I20260811 05:13:47.237942 18505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "95667e6974cb44ba9111427a619ddcef" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: NON_VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: true } } }
I20260811 05:13:47.238066 18505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:47.238404 18288 ts_manager.cc:194] Registered new tserver with Master: 32e63505878c446594d70821360ed6c5 (127.7.67.131:43549)
I20260811 05:13:47.245857 19301 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef: attempting to promote NON_VOTER 6f60f0e75bad4042a07087df578dfbdb to VOTER
I20260811 05:13:47.246259 19301 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: false } }
I20260811 05:13:47.247431 18304 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 1 FOLLOWER]: Refusing update from remote peer 95667e6974cb44ba9111427a619ddcef: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 10. (index mismatch)
I20260811 05:13:47.247859 19311 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 05:13:47.250512 18578 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 1 LEARNER]: Refusing update from remote peer 95667e6974cb44ba9111427a619ddcef: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 05:13:47.250981 19301 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 05:13:47.251305 19310 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6f60f0e75bad4042a07087df578dfbdb (127.7.67.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: false } } }
I20260811 05:13:47.251691 18304 raft_consensus.cc:2932] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6f60f0e75bad4042a07087df578dfbdb (127.7.67.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: false } } }
I20260811 05:13:47.252480 18578 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 6f60f0e75bad4042a07087df578dfbdb (127.7.67.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: false } } }
I20260811 05:13:47.255015 18620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "95667e6974cb44ba9111427a619ddcef" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: false } } }
I20260811 05:13:47.255102 18620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:13:47.255038 18505 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "95667e6974cb44ba9111427a619ddcef" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: false } } }
I20260811 05:13:47.255141 18505 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:47.256786 19311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "95667e6974cb44ba9111427a619ddcef" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: false } } }
I20260811 05:13:47.256896 19311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:47.257014 19314 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "95667e6974cb44ba9111427a619ddcef" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: false } } }
I20260811 05:13:47.257157 19314 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:13:47.262316 19320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.255927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:43549 (local address 127.0.0.1:49830)
0811 05:13:47.257388 (+ 1461us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:47.257402 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:47.257504 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:47.257965 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:47.257968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:47.257986 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:47.258104 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:47.258111 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.259404 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.259408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:47.260763 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:47.260774 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.261861 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.261865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:47.261960 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.261987 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 05:13:47.262074 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 05:13:47.262144 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1396,"thread_start_us":90,"threads_started":1}
I20260811 05:13:47.262326 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.256332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49830 (local address 127.7.67.131:43549)
0811 05:13:47.257539 (+ 1207us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:47.257555 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:47.257580 (+ 25us) server_negotiation.cc:407] Connection header received
0811 05:13:47.257657 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:47.257661 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:47.257828 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:47.257997 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:47.258226 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.259290 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.260903 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.261709 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.261824 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.261967 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:47.262100 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 05:13:47.262184 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":132,"threads_started":1}
I20260811 05:13:47.262699 19079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.255681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:36481 (local address 127.0.0.1:52496)
0811 05:13:47.255803 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:47.255819 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:47.255903 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:47.258497 (+ 2594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:47.258501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:47.258522 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:47.258621 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:47.258627 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.259995 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.260002 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:47.261213 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:47.261222 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.262374 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.262378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:47.262483 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.262507 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 05:13:47.262557 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 05:13:47.262620 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":27}
I20260811 05:13:47.263242 19319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.255881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52496 (local address 127.7.67.129:36481)
0811 05:13:47.258052 (+ 2171us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:47.258068 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:47.258090 (+ 22us) server_negotiation.cc:407] Connection header received
0811 05:13:47.258160 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:47.258165 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:47.258346 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:47.258526 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:47.258719 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.259861 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.261343 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.262217 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.262896 (+ 679us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.263023 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:47.263063 (+ 40us) server_negotiation.cc:299] Negotiation successful
0811 05:13:47.263128 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2069,"thread_start_us":63,"threads_started":1}
I20260811 05:13:47.264793 19321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.256279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:40403 (local address 127.0.0.1:40980)
0811 05:13:47.257682 (+ 1403us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:47.257693 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:47.257813 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:47.258286 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:47.258289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:47.258304 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:47.258408 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:47.258416 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.260086 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.260090 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:47.262326 (+ 2236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:47.262340 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.264386 (+ 2046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.264390 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:47.264479 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.264503 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 05:13:47.264585 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 05:13:47.264651 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1346,"thread_start_us":90,"threads_started":1}
I20260811 05:13:47.265409 19324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.256694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40980 (local address 127.7.67.130:40403)
0811 05:13:47.257436 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:47.257450 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:47.257798 (+ 348us) server_negotiation.cc:407] Connection header received
0811 05:13:47.257925 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:47.257930 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:47.258131 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:47.258615 (+ 484us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:47.258745 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.259954 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.263482 (+ 3528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.264228 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.264967 (+ 739us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.265126 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:47.265178 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 05:13:47.265278 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":121,"threads_started":1}
I20260811 05:13:47.265789 19182 tablet_service.cc:1467] Processing CreateTablet for tablet 1e16e2c354314312b0d62345022c3fef (DEFAULT_TABLE table=test-table [id=3e3ccf08736543a9b47aaf57fa4e11d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:47.265910 18734 tablet_service.cc:1467] Processing CreateTablet for tablet 1e16e2c354314312b0d62345022c3fef (DEFAULT_TABLE table=test-table [id=3e3ccf08736543a9b47aaf57fa4e11d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:47.266804 18734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e16e2c354314312b0d62345022c3fef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:47.266837 19182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e16e2c354314312b0d62345022c3fef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:47.268420 18964 tablet_service.cc:1467] Processing CreateTablet for tablet 1e16e2c354314312b0d62345022c3fef (DEFAULT_TABLE table=test-table [id=3e3ccf08736543a9b47aaf57fa4e11d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:47.269416 18964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e16e2c354314312b0d62345022c3fef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:47.274173 19326 tablet_bootstrap.cc:492] T 1e16e2c354314312b0d62345022c3fef P d2e72a180c5e43a5ba0728584cc25be4: Bootstrap starting.
I20260811 05:13:47.276973 19327 tablet_bootstrap.cc:492] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5: Bootstrap starting.
I20260811 05:13:47.277060 19328 tablet_bootstrap.cc:492] T 1e16e2c354314312b0d62345022c3fef P 82b986efd42f463ba147b7067465ac5e: Bootstrap starting.
I20260811 05:13:47.277014 19326 tablet_bootstrap.cc:654] T 1e16e2c354314312b0d62345022c3fef P d2e72a180c5e43a5ba0728584cc25be4: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:47.278105 19326 log.cc:824] T 1e16e2c354314312b0d62345022c3fef P d2e72a180c5e43a5ba0728584cc25be4: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:47.279192 19327 tablet_bootstrap.cc:654] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:47.279507 19328 tablet_bootstrap.cc:654] T 1e16e2c354314312b0d62345022c3fef P 82b986efd42f463ba147b7067465ac5e: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:47.279923 19327 log.cc:824] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:47.280309 19326 tablet_bootstrap.cc:492] T 1e16e2c354314312b0d62345022c3fef P d2e72a180c5e43a5ba0728584cc25be4: No bootstrap required, opened a new log
I20260811 05:13:47.280354 19328 log.cc:824] T 1e16e2c354314312b0d62345022c3fef P 82b986efd42f463ba147b7067465ac5e: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:47.280555 19326 ts_tablet_manager.cc:1397] T 1e16e2c354314312b0d62345022c3fef P d2e72a180c5e43a5ba0728584cc25be4: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.002s
I20260811 05:13:47.281733 19327 tablet_bootstrap.cc:492] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5: No bootstrap required, opened a new log
I20260811 05:13:47.281942 19327 ts_tablet_manager.cc:1397] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 05:13:47.282618 19328 tablet_bootstrap.cc:492] T 1e16e2c354314312b0d62345022c3fef P 82b986efd42f463ba147b7067465ac5e: No bootstrap required, opened a new log
I20260811 05:13:47.282914 19328 ts_tablet_manager.cc:1397] T 1e16e2c354314312b0d62345022c3fef P 82b986efd42f463ba147b7067465ac5e: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:13:47.285333 19326 raft_consensus.cc:348] T 1e16e2c354314312b0d62345022c3fef P d2e72a180c5e43a5ba0728584cc25be4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e72a180c5e43a5ba0728584cc25be4" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 36481 } } peers { permanent_uuid: "32e63505878c446594d70821360ed6c5" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43549 } } peers { permanent_uuid: "82b986efd42f463ba147b7067465ac5e" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 40403 } }
I20260811 05:13:47.285629 19326 raft_consensus.cc:374] T 1e16e2c354314312b0d62345022c3fef P d2e72a180c5e43a5ba0728584cc25be4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:47.285698 19326 raft_consensus.cc:729] T 1e16e2c354314312b0d62345022c3fef P d2e72a180c5e43a5ba0728584cc25be4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e72a180c5e43a5ba0728584cc25be4, State: Initialized, Role: FOLLOWER
I20260811 05:13:47.286202 19326 consensus_queue.cc:260] T 1e16e2c354314312b0d62345022c3fef P d2e72a180c5e43a5ba0728584cc25be4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e72a180c5e43a5ba0728584cc25be4" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 36481 } } peers { permanent_uuid: "32e63505878c446594d70821360ed6c5" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43549 } } peers { permanent_uuid: "82b986efd42f463ba147b7067465ac5e" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 40403 } }
I20260811 05:13:47.286952 19328 raft_consensus.cc:348] T 1e16e2c354314312b0d62345022c3fef P 82b986efd42f463ba147b7067465ac5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e72a180c5e43a5ba0728584cc25be4" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 36481 } } peers { permanent_uuid: "32e63505878c446594d70821360ed6c5" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43549 } } peers { permanent_uuid: "82b986efd42f463ba147b7067465ac5e" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 40403 } }
I20260811 05:13:47.287225 19328 raft_consensus.cc:374] T 1e16e2c354314312b0d62345022c3fef P 82b986efd42f463ba147b7067465ac5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:47.287288 19326 ts_tablet_manager.cc:1428] T 1e16e2c354314312b0d62345022c3fef P d2e72a180c5e43a5ba0728584cc25be4: Time spent starting tablet: real 0.007s user 0.000s sys 0.006s
I20260811 05:13:47.287294 19328 raft_consensus.cc:729] T 1e16e2c354314312b0d62345022c3fef P 82b986efd42f463ba147b7067465ac5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82b986efd42f463ba147b7067465ac5e, State: Initialized, Role: FOLLOWER
I20260811 05:13:47.287825 19328 consensus_queue.cc:260] T 1e16e2c354314312b0d62345022c3fef P 82b986efd42f463ba147b7067465ac5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e72a180c5e43a5ba0728584cc25be4" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 36481 } } peers { permanent_uuid: "32e63505878c446594d70821360ed6c5" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43549 } } peers { permanent_uuid: "82b986efd42f463ba147b7067465ac5e" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 40403 } }
I20260811 05:13:47.288884 19328 ts_tablet_manager.cc:1428] T 1e16e2c354314312b0d62345022c3fef P 82b986efd42f463ba147b7067465ac5e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 05:13:47.288767 19327 raft_consensus.cc:348] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e72a180c5e43a5ba0728584cc25be4" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 36481 } } peers { permanent_uuid: "32e63505878c446594d70821360ed6c5" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43549 } } peers { permanent_uuid: "82b986efd42f463ba147b7067465ac5e" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 40403 } }
I20260811 05:13:47.289086 19327 raft_consensus.cc:374] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:47.289186 19327 raft_consensus.cc:729] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32e63505878c446594d70821360ed6c5, State: Initialized, Role: FOLLOWER
I20260811 05:13:47.289242 19067 heartbeater.cc:503] Master 127.7.67.190:45385 was elected leader, sending a full tablet report...
I20260811 05:13:47.289685 19327 consensus_queue.cc:260] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e72a180c5e43a5ba0728584cc25be4" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 36481 } } peers { permanent_uuid: "32e63505878c446594d70821360ed6c5" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43549 } } peers { permanent_uuid: "82b986efd42f463ba147b7067465ac5e" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 40403 } }
I20260811 05:13:47.290000 18851 heartbeater.cc:503] Master 127.7.67.190:45385 was elected leader, sending a full tablet report...
I20260811 05:13:47.290727 19327 ts_tablet_manager.cc:1428] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260811 05:13:47.291500 19275 heartbeater.cc:503] Master 127.7.67.190:45385 was elected leader, sending a full tablet report...
I20260811 05:13:47.331945 19337 raft_consensus.cc:482] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:47.332237 19337 raft_consensus.cc:504] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e72a180c5e43a5ba0728584cc25be4" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 36481 } } peers { permanent_uuid: "32e63505878c446594d70821360ed6c5" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43549 } } peers { permanent_uuid: "82b986efd42f463ba147b7067465ac5e" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 40403 } }
I20260811 05:13:47.333500 19337 leader_election.cc:288] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2e72a180c5e43a5ba0728584cc25be4 (127.7.67.129:36481), 82b986efd42f463ba147b7067465ac5e (127.7.67.130:40403)
I20260811 05:13:47.338590 19319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.333858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:37349 (local address 127.7.67.129:36481)
0811 05:13:47.333985 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:47.333991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:47.334056 (+ 65us) server_negotiation.cc:407] Connection header received
0811 05:13:47.334164 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:47.334167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:47.334217 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:47.334296 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:47.334739 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.335519 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.337250 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.337880 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.338007 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.338060 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:47.338388 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 05:13:47.338462 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 05:13:47.338590 19324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.333947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:58297 (local address 127.7.67.130:40403)
0811 05:13:47.334061 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:47.334066 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:47.334080 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:13:47.334132 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:47.334136 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:47.334199 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:47.334293 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:47.334734 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.335520 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.337234 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.337880 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.338007 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.338060 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:47.338388 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 05:13:47.338464 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 05:13:47.338652 19282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.333764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:36481 (local address 127.7.67.131:37349)
0811 05:13:47.333997 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:47.334010 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:47.334086 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:47.334325 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:47.334328 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:47.334360 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:47.334604 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:47.334609 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.335639 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.335643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:47.337092 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:47.337104 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.338010 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.338014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:47.338227 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.338281 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 05:13:47.338354 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 05:13:47.338485 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":363,"spinlock_wait_cycles":7040}
I20260811 05:13:47.338652 19280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.333751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:40403 (local address 127.7.67.131:58297)
0811 05:13:47.333872 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:47.333889 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:47.333980 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:47.334325 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:47.334328 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:47.334361 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:47.334601 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:47.334609 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.335639 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.335643 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:47.337092 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:47.337104 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.338010 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.338014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:47.338224 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.338270 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 05:13:47.338354 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 05:13:47.338485 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":269,"spinlock_wait_cycles":7296}
I20260811 05:13:47.339342 18995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e16e2c354314312b0d62345022c3fef" candidate_uuid: "32e63505878c446594d70821360ed6c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82b986efd42f463ba147b7067465ac5e" is_pre_election: true
I20260811 05:13:47.339576 18765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e16e2c354314312b0d62345022c3fef" candidate_uuid: "32e63505878c446594d70821360ed6c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e72a180c5e43a5ba0728584cc25be4" is_pre_election: true
I20260811 05:13:47.339650 18995 raft_consensus.cc:2454] T 1e16e2c354314312b0d62345022c3fef P 82b986efd42f463ba147b7067465ac5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32e63505878c446594d70821360ed6c5 in term 0.
I20260811 05:13:47.339846 18765 raft_consensus.cc:2454] T 1e16e2c354314312b0d62345022c3fef P d2e72a180c5e43a5ba0728584cc25be4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32e63505878c446594d70821360ed6c5 in term 0.
I20260811 05:13:47.340153 19108 leader_election.cc:302] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32e63505878c446594d70821360ed6c5, 82b986efd42f463ba147b7067465ac5e; no voters:
I20260811 05:13:47.340415 19337 raft_consensus.cc:2781] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:47.340515 19337 raft_consensus.cc:482] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:47.340593 19337 raft_consensus.cc:3037] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:47.341738 19337 raft_consensus.cc:504] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e72a180c5e43a5ba0728584cc25be4" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 36481 } } peers { permanent_uuid: "32e63505878c446594d70821360ed6c5" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43549 } } peers { permanent_uuid: "82b986efd42f463ba147b7067465ac5e" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 40403 } }
I20260811 05:13:47.342065 19337 leader_election.cc:288] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5 [CANDIDATE]: Term 1 election: Requested vote from peers d2e72a180c5e43a5ba0728584cc25be4 (127.7.67.129:36481), 82b986efd42f463ba147b7067465ac5e (127.7.67.130:40403)
I20260811 05:13:47.342453 18765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e16e2c354314312b0d62345022c3fef" candidate_uuid: "32e63505878c446594d70821360ed6c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e72a180c5e43a5ba0728584cc25be4"
I20260811 05:13:47.342453 18995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e16e2c354314312b0d62345022c3fef" candidate_uuid: "32e63505878c446594d70821360ed6c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82b986efd42f463ba147b7067465ac5e"
I20260811 05:13:47.342581 18765 raft_consensus.cc:3037] T 1e16e2c354314312b0d62345022c3fef P d2e72a180c5e43a5ba0728584cc25be4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:47.342581 18995 raft_consensus.cc:3037] T 1e16e2c354314312b0d62345022c3fef P 82b986efd42f463ba147b7067465ac5e [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:47.343900 18765 raft_consensus.cc:2454] T 1e16e2c354314312b0d62345022c3fef P d2e72a180c5e43a5ba0728584cc25be4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32e63505878c446594d70821360ed6c5 in term 1.
I20260811 05:13:47.344234 19107 leader_election.cc:302] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32e63505878c446594d70821360ed6c5, d2e72a180c5e43a5ba0728584cc25be4; no voters:
I20260811 05:13:47.344419 18995 raft_consensus.cc:2454] T 1e16e2c354314312b0d62345022c3fef P 82b986efd42f463ba147b7067465ac5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32e63505878c446594d70821360ed6c5 in term 1.
I20260811 05:13:47.344506 19337 raft_consensus.cc:2781] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:47.344770 19337 raft_consensus.cc:686] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5 [term 1 LEADER]: Becoming Leader. State: Replica: 32e63505878c446594d70821360ed6c5, State: Running, Role: LEADER
I20260811 05:13:47.345130 19337 consensus_queue.cc:237] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e72a180c5e43a5ba0728584cc25be4" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 36481 } } peers { permanent_uuid: "32e63505878c446594d70821360ed6c5" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43549 } } peers { permanent_uuid: "82b986efd42f463ba147b7067465ac5e" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 40403 } }
I20260811 05:13:47.348531 17361 catalog_manager.cc:5495] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32e63505878c446594d70821360ed6c5 (127.7.67.131). New cstate: current_term: 1 leader_uuid: "32e63505878c446594d70821360ed6c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e72a180c5e43a5ba0728584cc25be4" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 36481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32e63505878c446594d70821360ed6c5" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "82b986efd42f463ba147b7067465ac5e" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 40403 } health_report { overall_health: UNKNOWN } } }
I20260811 05:13:47.350126 19301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "95667e6974cb44ba9111427a619ddcef" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: false } } }
I20260811 05:13:47.350311 19301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:47.385129 19322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.381283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49834 (local address 127.7.67.131:43549)
0811 05:13:47.381434 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:47.381441 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:47.381473 (+ 32us) server_negotiation.cc:407] Connection header received
0811 05:13:47.381694 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:47.381697 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:47.381770 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:47.381889 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:47.382142 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.382677 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.383555 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.383785 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.383901 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.384219 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:47.384688 (+ 469us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:47.384834 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:47.384895 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 05:13:47.384958 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 05:13:47.391510 18765 raft_consensus.cc:1261] T 1e16e2c354314312b0d62345022c3fef P d2e72a180c5e43a5ba0728584cc25be4 [term 1 FOLLOWER]: Refusing update from remote peer 32e63505878c446594d70821360ed6c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:47.391510 18995 raft_consensus.cc:1261] T 1e16e2c354314312b0d62345022c3fef P 82b986efd42f463ba147b7067465ac5e [term 1 FOLLOWER]: Refusing update from remote peer 32e63505878c446594d70821360ed6c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:47.392338 19338 consensus_queue.cc:1035] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82b986efd42f463ba147b7067465ac5e" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 40403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:47.392549 19337 consensus_queue.cc:1035] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e72a180c5e43a5ba0728584cc25be4" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 36481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:47.401854 19345 mvcc.cc:205] Tried to move back new op lower bound from 7317197731387576320 to 7317197731208269824. Current Snapshot: MvccSnapshot[applied={T|T < 7317197731387576320}]
I20260811 05:13:47.412400 17361 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:47.412631 17361 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57368:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:47.418591 17361 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 95667e6974cb44ba9111427a619ddcef: Sending DeleteTablet for 3 replicas of tablet 1e16e2c354314312b0d62345022c3fef
I20260811 05:13:47.419404 18734 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e16e2c354314312b0d62345022c3fef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:47 UTC) from {username='oryx', principal='oryx/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:52496
I20260811 05:13:47.419704 19182 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e16e2c354314312b0d62345022c3fef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:47 UTC) from {username='oryx', principal='oryx/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:49830
I20260811 05:13:47.420226 19362 tablet_replica.cc:330] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5: stopping tablet replica
I20260811 05:13:47.420645 19362 raft_consensus.cc:2229] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:13:47.421052 19362 raft_consensus.cc:2258] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:47.422097 18964 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e16e2c354314312b0d62345022c3fef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:47 UTC) from {username='oryx', principal='oryx/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:40980
I20260811 05:13:47.423031 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 17321
I20260811 05:13:47.423202 19362 ts_tablet_manager.cc:1905] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:47.434928 19362 ts_tablet_manager.cc:1918] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:47.435073 19362 log.cc:1196] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/1e16e2c354314312b0d62345022c3fef
I20260811 05:13:47.435468 19362 ts_tablet_manager.cc:1939] T 1e16e2c354314312b0d62345022c3fef P 32e63505878c446594d70821360ed6c5: Deleting consensus metadata
I20260811 05:13:47.440851 19366 tablet_replica.cc:330] T 1e16e2c354314312b0d62345022c3fef P 82b986efd42f463ba147b7067465ac5e: stopping tablet replica
I20260811 05:13:47.441277 19366 raft_consensus.cc:2229] T 1e16e2c354314312b0d62345022c3fef P 82b986efd42f463ba147b7067465ac5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:47.441676 18469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.421699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44312 (local address 127.7.67.189:40365)
0811 05:13:47.421811 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:47.421816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:47.421831 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:13:47.421870 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:47.421873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:47.421917 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:47.421992 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:47.433090 (+ 11098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.433926 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.435313 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.435664 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.435747 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.437908 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:47.437912 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:47.437916 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:47.437943 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:47.439683 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.440191 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.440195 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.440197 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.440283 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.440625 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.440628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.440630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.440845 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:47.440951 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:47.441388 (+ 437us) server_negotiation.cc:299] Negotiation successful
0811 05:13:47.441508 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 05:13:47.441715 19366 raft_consensus.cc:2258] T 1e16e2c354314312b0d62345022c3fef P 82b986efd42f463ba147b7067465ac5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:47.442231 19361 tablet_replica.cc:330] T 1e16e2c354314312b0d62345022c3fef P d2e72a180c5e43a5ba0728584cc25be4: stopping tablet replica
I20260811 05:13:47.442592 19361 raft_consensus.cc:2229] T 1e16e2c354314312b0d62345022c3fef P d2e72a180c5e43a5ba0728584cc25be4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:47.442950 19361 raft_consensus.cc:2258] T 1e16e2c354314312b0d62345022c3fef P d2e72a180c5e43a5ba0728584cc25be4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:47.443717 19366 ts_tablet_manager.cc:1905] T 1e16e2c354314312b0d62345022c3fef P 82b986efd42f463ba147b7067465ac5e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 05:13:47.444187 19111 connection.cc:535] server connection from 127.0.0.1:49830 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 05:13:47.444661 19361 ts_tablet_manager.cc:1905] T 1e16e2c354314312b0d62345022c3fef P d2e72a180c5e43a5ba0728584cc25be4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:47.446684 19366 ts_tablet_manager.cc:1918] T 1e16e2c354314312b0d62345022c3fef P 82b986efd42f463ba147b7067465ac5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:47.446847 19366 log.cc:1196] T 1e16e2c354314312b0d62345022c3fef P 82b986efd42f463ba147b7067465ac5e: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/1e16e2c354314312b0d62345022c3fef
I20260811 05:13:47.447732 19361 ts_tablet_manager.cc:1918] T 1e16e2c354314312b0d62345022c3fef P d2e72a180c5e43a5ba0728584cc25be4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:47.447850 19361 log.cc:1196] T 1e16e2c354314312b0d62345022c3fef P d2e72a180c5e43a5ba0728584cc25be4: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-0/wal/wals/1e16e2c354314312b0d62345022c3fef
I20260811 05:13:47.447956 19366 ts_tablet_manager.cc:1939] T 1e16e2c354314312b0d62345022c3fef P 82b986efd42f463ba147b7067465ac5e: Deleting consensus metadata
I20260811 05:13:47.448221 19361 ts_tablet_manager.cc:1939] T 1e16e2c354314312b0d62345022c3fef P d2e72a180c5e43a5ba0728584cc25be4: Deleting consensus metadata
W20260811 05:13:47.449462 18671 connection.cc:441] server connection from 127.0.0.1:52496 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:52496 (request call id 1) could send its response
W20260811 05:13:47.450366 18887 connection.cc:441] server connection from 127.0.0.1:40980 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40980 (request call id 1) could send its response
I20260811 05:13:47.459291 19297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.421244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55802 (local address 127.7.67.188:36501)
0811 05:13:47.421362 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:47.421369 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:47.447440 (+ 26071us) server_negotiation.cc:407] Connection header received
0811 05:13:47.448387 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:47.448391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:47.448448 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:47.448505 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:47.450630 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.451462 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.452749 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.453113 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.453241 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.455064 (+ 1823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:47.455068 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:47.455071 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:47.455098 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:47.457085 (+ 1987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.457875 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.457879 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.457881 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.457966 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.458270 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.458273 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.458275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.458479 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:47.458587 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:47.458942 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 05:13:47.459101 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":46,"spinlock_wait_cycles":1280}
I20260811 05:13:47.461421 18862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.450520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55814 (local address 127.7.67.188:36501)
0811 05:13:47.450671 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:47.450677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:47.450881 (+ 204us) server_negotiation.cc:407] Connection header received
0811 05:13:47.451085 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:47.451089 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:47.451146 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:47.451207 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:47.452759 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.453523 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.454593 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.455004 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.455137 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.457017 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:47.457021 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:47.457024 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:47.457051 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:47.459399 (+ 2348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.459924 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.459929 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.459931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.460024 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.460354 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.460359 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.460361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.460598 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:47.460761 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:47.461218 (+ 457us) server_negotiation.cc:299] Negotiation successful
0811 05:13:47.461312 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:13:47.463793 18469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:47.451185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44320 (local address 127.7.67.189:40365)
0811 05:13:47.451320 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:47.451325 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:47.452975 (+ 1650us) server_negotiation.cc:407] Connection header received
0811 05:13:47.453158 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:47.453162 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:47.453208 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:47.453263 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:47.455308 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.456059 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.457369 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:47.457775 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:47.457878 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:47.459833 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:47.459836 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:47.459839 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:47.459866 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:47.461726 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.462329 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.462333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.462335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.462439 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:47.462820 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:47.462824 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:47.462826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:47.463057 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:47.463164 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:47.463522 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 05:13:47.463638 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:13:48.407255 19375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:48.406761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.130:53533)
0811 05:13:48.407003 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:48.407089 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":71,"threads_started":1}
W20260811 05:13:48.407636 19067 heartbeater.cc:644] Failed to heartbeat to 127.7.67.190:45385 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
I20260811 05:13:48.410657 19376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:48.410102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.129:48969)
0811 05:13:48.410428 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:48.410521 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":136,"threads_started":1}
W20260811 05:13:48.411056 18851 heartbeater.cc:644] Failed to heartbeat to 127.7.67.190:45385 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
I20260811 05:13:48.420017 19377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:48.419594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.131:48193)
0811 05:13:48.419858 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:48.419936 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":84,"threads_started":1}
W20260811 05:13:48.420346 19275 heartbeater.cc:644] Failed to heartbeat to 127.7.67.190:45385 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
I20260811 05:13:48.918237 19379 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 95667e6974cb44ba9111427a619ddcef)
I20260811 05:13:48.918438 19380 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 95667e6974cb44ba9111427a619ddcef)
I20260811 05:13:48.918503 19379 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: false } }
I20260811 05:13:48.918638 19380 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: false } }
I20260811 05:13:48.919639 19380 leader_election.cc:288] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 95667e6974cb44ba9111427a619ddcef (127.7.67.190:45385), 6f60f0e75bad4042a07087df578dfbdb (127.7.67.188:36501)
I20260811 05:13:48.920019 19379 leader_election.cc:288] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 95667e6974cb44ba9111427a619ddcef (127.7.67.190:45385), ba069f98f23c4f3b9125de6627e8992d (127.7.67.189:40365)
I20260811 05:13:48.920310 19382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:48.919803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.0.0.1:57828)
0811 05:13:48.920118 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:48.920202 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":53,"threads_started":1}
I20260811 05:13:48.920424 19384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:48.919844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.0.0.1:57826)
0811 05:13:48.920237 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:48.920312 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":52,"threads_started":1}
W20260811 05:13:48.920727 18541 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
W20260811 05:13:48.920758 18266 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
I20260811 05:13:48.921749 19384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:48.921462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.0.0.1:57840)
0811 05:13:48.921610 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:48.921658 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 05:13:48.921890 19388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:48.921640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.0.0.1:57850)
0811 05:13:48.921762 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:48.921822 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 05:13:48.922000 18266 leader_election.cc:334] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 95667e6974cb44ba9111427a619ddcef (127.7.67.190:45385): Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
W20260811 05:13:48.922024 18541 leader_election.cc:334] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 95667e6974cb44ba9111427a619ddcef (127.7.67.190:45385): Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
I20260811 05:13:48.925307 19382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:48.920132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:36501 (local address 127.0.0.1:55822)
0811 05:13:48.920465 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:48.920479 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:48.920611 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:48.921112 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:48.921116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:48.921163 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:48.921331 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:48.921338 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:48.922819 (+ 1481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:48.922822 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:48.924032 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:48.924044 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:48.924936 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:48.924940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:48.925042 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:48.925072 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 05:13:48.925125 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 05:13:48.925200 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":42,"spinlock_wait_cycles":896,"thread_start_us":84,"threads_started":1}
I20260811 05:13:48.925719 19386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:48.919896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44328 (local address 127.7.67.189:40365)
0811 05:13:48.920685 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:48.920690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:48.920708 (+ 18us) server_negotiation.cc:407] Connection header received
0811 05:13:48.920992 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:48.920998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:48.921048 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:48.921105 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:48.921575 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:48.922351 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:48.923879 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:48.925127 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:48.925278 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:48.925335 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:48.925524 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 05:13:48.925593 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":717,"spinlock_wait_cycles":23680,"thread_start_us":46,"threads_started":1}
I20260811 05:13:48.925729 19383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:48.919820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:40365 (local address 127.0.0.1:44328)
0811 05:13:48.920199 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:48.920215 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:48.920851 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:48.921331 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:48.921333 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:48.921352 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:48.921456 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:48.921461 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:48.922473 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:48.922476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:48.923695 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:48.923707 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:48.925278 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:48.925282 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:48.925383 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:48.925428 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 05:13:48.925504 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 05:13:48.925583 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":74,"threads_started":1}
I20260811 05:13:48.925845 19390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:48.920505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55822 (local address 127.7.67.188:36501)
0811 05:13:48.920851 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:48.920856 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:48.920867 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:13:48.920919 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:48.920922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:48.920969 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:48.921201 (+ 232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:48.921454 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:48.922635 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:48.924180 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:48.924798 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:48.925520 (+ 722us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:48.925578 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:48.925611 (+ 33us) server_negotiation.cc:299] Negotiation successful
0811 05:13:48.925686 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":66,"threads_started":1}
I20260811 05:13:48.926232 18578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ba069f98f23c4f3b9125de6627e8992d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6f60f0e75bad4042a07087df578dfbdb" is_pre_election: true
I20260811 05:13:48.926244 18304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f60f0e75bad4042a07087df578dfbdb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ba069f98f23c4f3b9125de6627e8992d" is_pre_election: true
I20260811 05:13:48.926433 18578 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba069f98f23c4f3b9125de6627e8992d in term 1.
I20260811 05:13:48.926433 18304 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f60f0e75bad4042a07087df578dfbdb in term 1.
I20260811 05:13:48.926759 18266 leader_election.cc:302] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f60f0e75bad4042a07087df578dfbdb, ba069f98f23c4f3b9125de6627e8992d; no voters: 95667e6974cb44ba9111427a619ddcef
I20260811 05:13:48.926792 18540 leader_election.cc:302] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6f60f0e75bad4042a07087df578dfbdb, ba069f98f23c4f3b9125de6627e8992d; no voters: 95667e6974cb44ba9111427a619ddcef
I20260811 05:13:48.926970 19380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 05:13:48.927029 19380 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 1 FOLLOWER]: Starting leader election (detected failure of leader 95667e6974cb44ba9111427a619ddcef)
I20260811 05:13:48.927026 19379 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 05:13:48.927069 19380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 1 FOLLOWER]: Advancing to term 2
I20260811 05:13:48.927083 19379 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 1 FOLLOWER]: Starting leader election (detected failure of leader 95667e6974cb44ba9111427a619ddcef)
I20260811 05:13:48.927122 19379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 1 FOLLOWER]: Advancing to term 2
I20260811 05:13:48.928104 19380 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: false } }
I20260811 05:13:48.928102 19379 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: false } }
I20260811 05:13:48.928404 19380 leader_election.cc:288] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [CANDIDATE]: Term 2 election: Requested vote from peers 95667e6974cb44ba9111427a619ddcef (127.7.67.190:45385), 6f60f0e75bad4042a07087df578dfbdb (127.7.67.188:36501)
I20260811 05:13:48.928442 19379 leader_election.cc:288] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [CANDIDATE]: Term 2 election: Requested vote from peers 95667e6974cb44ba9111427a619ddcef (127.7.67.190:45385), ba069f98f23c4f3b9125de6627e8992d (127.7.67.189:40365)
I20260811 05:13:48.928841 19383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:48.928586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.0.0.1:57864)
0811 05:13:48.928731 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:48.928781 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260811 05:13:48.928841 18304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6f60f0e75bad4042a07087df578dfbdb" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "ba069f98f23c4f3b9125de6627e8992d"
I20260811 05:13:48.928915 18578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ba069f98f23c4f3b9125de6627e8992d" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6f60f0e75bad4042a07087df578dfbdb"
I20260811 05:13:48.928989 18304 raft_consensus.cc:2379] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6f60f0e75bad4042a07087df578dfbdb in current term 2: Already voted for candidate ba069f98f23c4f3b9125de6627e8992d in this term.
I20260811 05:13:48.929082 18578 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate ba069f98f23c4f3b9125de6627e8992d in current term 2: Already voted for candidate 6f60f0e75bad4042a07087df578dfbdb in this term.
I20260811 05:13:48.929323 19382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:48.928554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.0.0.1:57860)
0811 05:13:48.928771 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:48.929265 (+ 494us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260811 05:13:48.929450 19384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:48.929249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.0.0.1:57868)
0811 05:13:48.929338 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:48.929387 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260811 05:13:48.929565 18541 leader_election.cc:334] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 95667e6974cb44ba9111427a619ddcef (127.7.67.190:45385): Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
I20260811 05:13:48.929682 18541 leader_election.cc:302] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6f60f0e75bad4042a07087df578dfbdb; no voters: 95667e6974cb44ba9111427a619ddcef, ba069f98f23c4f3b9125de6627e8992d
I20260811 05:13:48.929844 19379 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 05:13:48.930212 19382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:48.930017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.0.0.1:57878)
0811 05:13:48.930102 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:48.930153 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260811 05:13:48.930322 18266 leader_election.cc:334] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 95667e6974cb44ba9111427a619ddcef (127.7.67.190:45385): Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
I20260811 05:13:48.930395 18266 leader_election.cc:302] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ba069f98f23c4f3b9125de6627e8992d; no voters: 6f60f0e75bad4042a07087df578dfbdb, 95667e6974cb44ba9111427a619ddcef
I20260811 05:13:48.930516 19380 raft_consensus.cc:2726] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 05:13:49.408768 19393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:49.408236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.130:37179)
0811 05:13:49.408597 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:49.408667 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":95,"threads_started":1}
I20260811 05:13:49.412310 19394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:49.411846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.129:55877)
0811 05:13:49.412124 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:49.412193 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":84,"threads_started":1}
I20260811 05:13:49.421280 19395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:49.420888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.131:52995)
0811 05:13:49.421120 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:49.421192 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":62,"threads_started":1}
I20260811 05:13:50.410074 19396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.409610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.130:45677)
0811 05:13:50.409857 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.409953 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":85,"threads_started":1}
I20260811 05:13:50.413699 19397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.413075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.129:57269)
0811 05:13:50.413506 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.413571 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":120,"threads_started":1}
I20260811 05:13:50.422322 19398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.421888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.131:42871)
0811 05:13:50.422117 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.422241 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":87,"threads_started":1}
I20260811 05:13:50.646584 19400 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:50.646718 19400 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: false } }
I20260811 05:13:50.647230 19400 leader_election.cc:288] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 95667e6974cb44ba9111427a619ddcef (127.7.67.190:45385), 6f60f0e75bad4042a07087df578dfbdb (127.7.67.188:36501)
I20260811 05:13:50.648022 18578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ba069f98f23c4f3b9125de6627e8992d" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6f60f0e75bad4042a07087df578dfbdb" is_pre_election: true
I20260811 05:13:50.648072 19401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.647462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.0.0.1:58072)
0811 05:13:50.647849 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.647971 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":63,"threads_started":1}
I20260811 05:13:50.648182 18578 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ba069f98f23c4f3b9125de6627e8992d in term 2.
I20260811 05:13:50.648602 18266 leader_election.cc:302] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f60f0e75bad4042a07087df578dfbdb, ba069f98f23c4f3b9125de6627e8992d; no voters:
I20260811 05:13:50.648847 19400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 05:13:50.648919 19400 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:50.648973 19400 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 2 FOLLOWER]: Advancing to term 3
I20260811 05:13:50.649140 19401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.648889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.0.0.1:58084)
0811 05:13:50.649043 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.649087 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260811 05:13:50.649246 18266 leader_election.cc:334] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 95667e6974cb44ba9111427a619ddcef (127.7.67.190:45385): Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
I20260811 05:13:50.650241 19400 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: false } }
I20260811 05:13:50.650600 19400 leader_election.cc:288] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [CANDIDATE]: Term 3 election: Requested vote from peers 95667e6974cb44ba9111427a619ddcef (127.7.67.190:45385), 6f60f0e75bad4042a07087df578dfbdb (127.7.67.188:36501)
I20260811 05:13:50.650979 19401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.650744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.0.0.1:58092)
0811 05:13:50.650862 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.650916 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 05:13:50.651146 18578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ba069f98f23c4f3b9125de6627e8992d" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "6f60f0e75bad4042a07087df578dfbdb"
I20260811 05:13:50.651243 18578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 2 FOLLOWER]: Advancing to term 3
I20260811 05:13:50.652128 18578 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ba069f98f23c4f3b9125de6627e8992d in term 3.
I20260811 05:13:50.652505 18266 leader_election.cc:302] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f60f0e75bad4042a07087df578dfbdb, ba069f98f23c4f3b9125de6627e8992d; no voters:
I20260811 05:13:50.652747 19400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 3 FOLLOWER]: Leader election won for term 3
I20260811 05:13:50.652979 19400 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [term 3 LEADER]: Becoming Leader. State: Replica: ba069f98f23c4f3b9125de6627e8992d, State: Running, Role: LEADER
I20260811 05:13:50.653012 19401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.652801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.0.0.1:58106)
0811 05:13:50.652901 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.652952 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 05:13:50.653185 18266 leader_election.cc:334] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 95667e6974cb44ba9111427a619ddcef (127.7.67.190:45385): Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
I20260811 05:13:50.653327 19400 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: false } }
I20260811 05:13:50.654479 19403 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba069f98f23c4f3b9125de6627e8992d. Latest consensus state: current_term: 3 leader_uuid: "ba069f98f23c4f3b9125de6627e8992d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: false } } }
I20260811 05:13:50.654572 19403 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:50.654871 19406 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:50.657349 19406 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:50.657776 19406 catalog_manager.cc:1259] Loaded cluster ID: e9f81071b52b4f468d783cb68b56dd85
I20260811 05:13:50.657823 19406 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:50.658396 19406 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:50.658809 19406 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d: Loaded TSK: 0
I20260811 05:13:50.659813 19406 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:13:50.683197 18288 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44320:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:13:50.686313 18288 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:50.689535 19401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.689265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.0.0.1:58132)
0811 05:13:50.689411 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.689463 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 05:13:50.689570 18578 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [term 3 FOLLOWER]: Refusing update from remote peer ba069f98f23c4f3b9125de6627e8992d: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 05:13:50.690092 19400 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 05:13:50.690387 19401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.690152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.0.0.1:58144)
0811 05:13:50.690273 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.690330 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 05:13:50.690505 18266 consensus_peers.cc:487] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d -> Peer 95667e6974cb44ba9111427a619ddcef (127.7.67.190:45385): Couldn't send request to peer 95667e6974cb44ba9111427a619ddcef. Status: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 05:13:50.691650 19411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba069f98f23c4f3b9125de6627e8992d. Latest consensus state: current_term: 3 leader_uuid: "ba069f98f23c4f3b9125de6627e8992d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: false } } }
I20260811 05:13:50.691777 19411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:13:50.693377 19400 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "ba069f98f23c4f3b9125de6627e8992d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: false } } }
I20260811 05:13:50.693511 19400 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:50.694051 19411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "ba069f98f23c4f3b9125de6627e8992d" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "95667e6974cb44ba9111427a619ddcef" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45385 } } peers { permanent_uuid: "ba069f98f23c4f3b9125de6627e8992d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 40365 } attrs { promote: false } } peers { permanent_uuid: "6f60f0e75bad4042a07087df578dfbdb" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 36501 } attrs { promote: false } } }
I20260811 05:13:50.694178 19411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f60f0e75bad4042a07087df578dfbdb [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:13:50.705163 19429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.700924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49844 (local address 127.7.67.131:43549)
0811 05:13:50.701183 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:50.701188 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:50.701522 (+ 334us) server_negotiation.cc:407] Connection header received
0811 05:13:50.701617 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:50.701620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:50.701669 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:50.701758 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:50.702039 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:50.702671 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:50.703933 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:50.704584 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:50.704711 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:50.704781 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:50.704965 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 05:13:50.705047 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":83,"threads_started":1}
I20260811 05:13:50.705164 19427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.700445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:43549 (local address 127.0.0.1:49844)
0811 05:13:50.701409 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.701426 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:50.701526 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:50.701785 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:50.701789 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:50.701809 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:50.701912 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:50.701919 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:50.702779 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:50.702783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:50.703792 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:50.703803 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:50.704723 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:50.704727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:50.704830 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:50.704858 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 05:13:50.704946 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 05:13:50.705018 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":901,"mutex_wait_us":39,"thread_start_us":141,"threads_started":1}
I20260811 05:13:50.705619 19401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.699892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:36481 (local address 127.0.0.1:52498)
0811 05:13:50.700355 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.700374 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:50.700469 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:50.701123 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:50.701127 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:50.701144 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:50.701278 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:50.701284 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:50.702464 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:50.702468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:50.704163 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:50.704174 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:50.705189 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:50.705193 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:50.705295 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:50.705321 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 05:13:50.705378 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 05:13:50.705466 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":268}
I20260811 05:13:50.705663 19424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.700138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:40403 (local address 127.0.0.1:40984)
0811 05:13:50.700455 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.700469 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:50.700584 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:50.701225 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:50.701231 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:50.701254 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:50.701369 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:50.701376 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:50.702516 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:50.702519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:50.704162 (+ 1643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:50.704174 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:50.705273 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:50.705276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:50.705361 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:50.705390 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 05:13:50.705466 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 05:13:50.705525 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":222,"thread_start_us":102,"threads_started":1}
I20260811 05:13:50.705708 19428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.700569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40984 (local address 127.7.67.130:40403)
0811 05:13:50.700845 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:50.700852 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:50.700870 (+ 18us) server_negotiation.cc:407] Connection header received
0811 05:13:50.700930 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:50.700933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:50.700986 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:50.701085 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:50.701576 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:50.702367 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:50.704336 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:50.705021 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:50.705142 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:50.705217 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:50.705486 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 05:13:50.705569 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":75,"threads_started":1}
I20260811 05:13:50.706064 19182 tablet_service.cc:1467] Processing CreateTablet for tablet e1734f079c4744ff876530006eebca64 (DEFAULT_TABLE table=test-table [id=e2444dc97c074f2aa1ab42e8ec06cab3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:50.706288 19421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.700097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52498 (local address 127.7.67.129:36481)
0811 05:13:50.700853 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:50.700858 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:50.700873 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:13:50.700931 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:50.700934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:50.700984 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:50.701414 (+ 430us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:50.701546 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:50.702315 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:50.704336 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:50.705011 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:50.705936 (+ 925us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:50.706008 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:50.706049 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 05:13:50.706146 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":90,"threads_started":1}
I20260811 05:13:50.706323 18964 tablet_service.cc:1467] Processing CreateTablet for tablet e1734f079c4744ff876530006eebca64 (DEFAULT_TABLE table=test-table [id=e2444dc97c074f2aa1ab42e8ec06cab3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:50.706408 19182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1734f079c4744ff876530006eebca64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:50.706620 18964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1734f079c4744ff876530006eebca64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:50.708693 19431 tablet_bootstrap.cc:492] T e1734f079c4744ff876530006eebca64 P 82b986efd42f463ba147b7067465ac5e: Bootstrap starting.
I20260811 05:13:50.709220 19432 tablet_bootstrap.cc:492] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5: Bootstrap starting.
I20260811 05:13:50.709192 18734 tablet_service.cc:1467] Processing CreateTablet for tablet e1734f079c4744ff876530006eebca64 (DEFAULT_TABLE table=test-table [id=e2444dc97c074f2aa1ab42e8ec06cab3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:50.709492 18734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1734f079c4744ff876530006eebca64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:50.709811 19431 tablet_bootstrap.cc:654] T e1734f079c4744ff876530006eebca64 P 82b986efd42f463ba147b7067465ac5e: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:50.710310 19432 tablet_bootstrap.cc:654] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:50.711248 19431 tablet_bootstrap.cc:492] T e1734f079c4744ff876530006eebca64 P 82b986efd42f463ba147b7067465ac5e: No bootstrap required, opened a new log
I20260811 05:13:50.711354 19431 ts_tablet_manager.cc:1397] T e1734f079c4744ff876530006eebca64 P 82b986efd42f463ba147b7067465ac5e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:13:50.711781 19432 tablet_bootstrap.cc:492] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5: No bootstrap required, opened a new log
I20260811 05:13:50.711865 19432 ts_tablet_manager.cc:1397] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:13:50.711916 19431 raft_consensus.cc:348] T e1734f079c4744ff876530006eebca64 P 82b986efd42f463ba147b7067465ac5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e72a180c5e43a5ba0728584cc25be4" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 36481 } } peers { permanent_uuid: "82b986efd42f463ba147b7067465ac5e" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 40403 } } peers { permanent_uuid: "32e63505878c446594d70821360ed6c5" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43549 } }
I20260811 05:13:50.712031 19431 raft_consensus.cc:374] T e1734f079c4744ff876530006eebca64 P 82b986efd42f463ba147b7067465ac5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:50.712067 19431 raft_consensus.cc:729] T e1734f079c4744ff876530006eebca64 P 82b986efd42f463ba147b7067465ac5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82b986efd42f463ba147b7067465ac5e, State: Initialized, Role: FOLLOWER
I20260811 05:13:50.712143 19436 tablet_bootstrap.cc:492] T e1734f079c4744ff876530006eebca64 P d2e72a180c5e43a5ba0728584cc25be4: Bootstrap starting.
I20260811 05:13:50.712225 19431 consensus_queue.cc:260] T e1734f079c4744ff876530006eebca64 P 82b986efd42f463ba147b7067465ac5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e72a180c5e43a5ba0728584cc25be4" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 36481 } } peers { permanent_uuid: "82b986efd42f463ba147b7067465ac5e" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 40403 } } peers { permanent_uuid: "32e63505878c446594d70821360ed6c5" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43549 } }
I20260811 05:13:50.712368 19432 raft_consensus.cc:348] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e72a180c5e43a5ba0728584cc25be4" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 36481 } } peers { permanent_uuid: "82b986efd42f463ba147b7067465ac5e" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 40403 } } peers { permanent_uuid: "32e63505878c446594d70821360ed6c5" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43549 } }
I20260811 05:13:50.712489 19432 raft_consensus.cc:374] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:50.712553 19432 raft_consensus.cc:729] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32e63505878c446594d70821360ed6c5, State: Initialized, Role: FOLLOWER
I20260811 05:13:50.712677 19431 ts_tablet_manager.cc:1428] T e1734f079c4744ff876530006eebca64 P 82b986efd42f463ba147b7067465ac5e: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 05:13:50.712738 19432 consensus_queue.cc:260] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e72a180c5e43a5ba0728584cc25be4" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 36481 } } peers { permanent_uuid: "82b986efd42f463ba147b7067465ac5e" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 40403 } } peers { permanent_uuid: "32e63505878c446594d70821360ed6c5" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43549 } }
I20260811 05:13:50.713150 19432 ts_tablet_manager.cc:1428] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260811 05:13:50.713500 19396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.713177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.130:58953)
0811 05:13:50.713343 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.713415 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 05:13:50.713924 19398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.713562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.131:38723)
0811 05:13:50.713792 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.713842 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165}
I20260811 05:13:50.714924 19436 tablet_bootstrap.cc:654] T e1734f079c4744ff876530006eebca64 P d2e72a180c5e43a5ba0728584cc25be4: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:50.716236 19436 tablet_bootstrap.cc:492] T e1734f079c4744ff876530006eebca64 P d2e72a180c5e43a5ba0728584cc25be4: No bootstrap required, opened a new log
I20260811 05:13:50.716343 19436 ts_tablet_manager.cc:1397] T e1734f079c4744ff876530006eebca64 P d2e72a180c5e43a5ba0728584cc25be4: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 05:13:50.716485 19396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.716208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.130:54989)
0811 05:13:50.716360 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.716421 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 05:13:50.716794 19069 heartbeater.cc:503] Master 127.7.67.189:40365 was elected leader, sending a full tablet report...
I20260811 05:13:50.717017 19278 heartbeater.cc:503] Master 127.7.67.189:40365 was elected leader, sending a full tablet report...
I20260811 05:13:50.717224 19398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.716870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.131:51345)
0811 05:13:50.717087 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.717146 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":154}
I20260811 05:13:50.718176 19436 raft_consensus.cc:348] T e1734f079c4744ff876530006eebca64 P d2e72a180c5e43a5ba0728584cc25be4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e72a180c5e43a5ba0728584cc25be4" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 36481 } } peers { permanent_uuid: "82b986efd42f463ba147b7067465ac5e" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 40403 } } peers { permanent_uuid: "32e63505878c446594d70821360ed6c5" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43549 } }
I20260811 05:13:50.718334 19436 raft_consensus.cc:374] T e1734f079c4744ff876530006eebca64 P d2e72a180c5e43a5ba0728584cc25be4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:50.718400 19436 raft_consensus.cc:729] T e1734f079c4744ff876530006eebca64 P d2e72a180c5e43a5ba0728584cc25be4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2e72a180c5e43a5ba0728584cc25be4, State: Initialized, Role: FOLLOWER
I20260811 05:13:50.718536 19436 consensus_queue.cc:260] T e1734f079c4744ff876530006eebca64 P d2e72a180c5e43a5ba0728584cc25be4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e72a180c5e43a5ba0728584cc25be4" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 36481 } } peers { permanent_uuid: "82b986efd42f463ba147b7067465ac5e" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 40403 } } peers { permanent_uuid: "32e63505878c446594d70821360ed6c5" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43549 } }
I20260811 05:13:50.718924 19436 ts_tablet_manager.cc:1428] T e1734f079c4744ff876530006eebca64 P d2e72a180c5e43a5ba0728584cc25be4: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:13:50.719503 19397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.719239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.129:43745)
0811 05:13:50.719352 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.719428 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 05:13:50.720170 19397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.719923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.129:46973)
0811 05:13:50.720053 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.720114 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 05:13:50.720570 18854 heartbeater.cc:503] Master 127.7.67.189:40365 was elected leader, sending a full tablet report...
W20260811 05:13:50.737397 19071 tablet.cc:2369] T e1734f079c4744ff876530006eebca64 P 82b986efd42f463ba147b7067465ac5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:13:50.763761 18858 tablet.cc:2369] T e1734f079c4744ff876530006eebca64 P d2e72a180c5e43a5ba0728584cc25be4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:13:50.771098 19438 raft_consensus.cc:482] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:50.771268 19438 raft_consensus.cc:504] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e72a180c5e43a5ba0728584cc25be4" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 36481 } } peers { permanent_uuid: "82b986efd42f463ba147b7067465ac5e" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 40403 } } peers { permanent_uuid: "32e63505878c446594d70821360ed6c5" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43549 } }
I20260811 05:13:50.771742 19438 leader_election.cc:288] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d2e72a180c5e43a5ba0728584cc25be4 (127.7.67.129:36481), 82b986efd42f463ba147b7067465ac5e (127.7.67.130:40403)
I20260811 05:13:50.772274 18765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1734f079c4744ff876530006eebca64" candidate_uuid: "32e63505878c446594d70821360ed6c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e72a180c5e43a5ba0728584cc25be4" is_pre_election: true
I20260811 05:13:50.772274 18995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1734f079c4744ff876530006eebca64" candidate_uuid: "32e63505878c446594d70821360ed6c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82b986efd42f463ba147b7067465ac5e" is_pre_election: true
I20260811 05:13:50.772434 18765 raft_consensus.cc:2454] T e1734f079c4744ff876530006eebca64 P d2e72a180c5e43a5ba0728584cc25be4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32e63505878c446594d70821360ed6c5 in term 0.
I20260811 05:13:50.772434 18995 raft_consensus.cc:2454] T e1734f079c4744ff876530006eebca64 P 82b986efd42f463ba147b7067465ac5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32e63505878c446594d70821360ed6c5 in term 0.
I20260811 05:13:50.772842 19108 leader_election.cc:302] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 32e63505878c446594d70821360ed6c5, 82b986efd42f463ba147b7067465ac5e, d2e72a180c5e43a5ba0728584cc25be4; no voters:
I20260811 05:13:50.773061 19438 raft_consensus.cc:2781] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:50.773133 19438 raft_consensus.cc:482] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:50.773237 19438 raft_consensus.cc:3037] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:50.774361 19438 raft_consensus.cc:504] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e72a180c5e43a5ba0728584cc25be4" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 36481 } } peers { permanent_uuid: "82b986efd42f463ba147b7067465ac5e" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 40403 } } peers { permanent_uuid: "32e63505878c446594d70821360ed6c5" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43549 } }
I20260811 05:13:50.774731 19438 leader_election.cc:288] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5 [CANDIDATE]: Term 1 election: Requested vote from peers d2e72a180c5e43a5ba0728584cc25be4 (127.7.67.129:36481), 82b986efd42f463ba147b7067465ac5e (127.7.67.130:40403)
I20260811 05:13:50.775095 18995 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1734f079c4744ff876530006eebca64" candidate_uuid: "32e63505878c446594d70821360ed6c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82b986efd42f463ba147b7067465ac5e"
I20260811 05:13:50.775122 18765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1734f079c4744ff876530006eebca64" candidate_uuid: "32e63505878c446594d70821360ed6c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d2e72a180c5e43a5ba0728584cc25be4"
I20260811 05:13:50.775205 18995 raft_consensus.cc:3037] T e1734f079c4744ff876530006eebca64 P 82b986efd42f463ba147b7067465ac5e [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:50.775288 18765 raft_consensus.cc:3037] T e1734f079c4744ff876530006eebca64 P d2e72a180c5e43a5ba0728584cc25be4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:50.776190 18995 raft_consensus.cc:2454] T e1734f079c4744ff876530006eebca64 P 82b986efd42f463ba147b7067465ac5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32e63505878c446594d70821360ed6c5 in term 1.
I20260811 05:13:50.776314 18765 raft_consensus.cc:2454] T e1734f079c4744ff876530006eebca64 P d2e72a180c5e43a5ba0728584cc25be4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32e63505878c446594d70821360ed6c5 in term 1.
I20260811 05:13:50.776515 19108 leader_election.cc:302] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32e63505878c446594d70821360ed6c5, 82b986efd42f463ba147b7067465ac5e; no voters:
I20260811 05:13:50.776732 19438 raft_consensus.cc:2781] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:50.776978 19438 raft_consensus.cc:686] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5 [term 1 LEADER]: Becoming Leader. State: Replica: 32e63505878c446594d70821360ed6c5, State: Running, Role: LEADER
I20260811 05:13:50.777149 19438 consensus_queue.cc:237] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e72a180c5e43a5ba0728584cc25be4" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 36481 } } peers { permanent_uuid: "82b986efd42f463ba147b7067465ac5e" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 40403 } } peers { permanent_uuid: "32e63505878c446594d70821360ed6c5" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43549 } }
I20260811 05:13:50.777801 19398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.777491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.131:50475)
0811 05:13:50.777593 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.777690 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 05:13:50.779584 18285 catalog_manager.cc:5495] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32e63505878c446594d70821360ed6c5 (127.7.67.131). New cstate: current_term: 1 leader_uuid: "32e63505878c446594d70821360ed6c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d2e72a180c5e43a5ba0728584cc25be4" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 36481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82b986efd42f463ba147b7067465ac5e" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 40403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32e63505878c446594d70821360ed6c5" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43549 } health_report { overall_health: HEALTHY } } }
I20260811 05:13:50.807469 19429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.804188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49848 (local address 127.7.67.131:43549)
0811 05:13:50.804326 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:50.804329 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:50.804344 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:13:50.804393 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:50.804396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:50.804485 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:50.804623 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:50.804950 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:50.805526 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:50.806367 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:50.806607 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:50.806721 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:50.807044 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:50.807119 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:50.807180 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:50.807299 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 05:13:50.807356 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 05:13:50.810024 18995 raft_consensus.cc:1261] T e1734f079c4744ff876530006eebca64 P 82b986efd42f463ba147b7067465ac5e [term 1 FOLLOWER]: Refusing update from remote peer 32e63505878c446594d70821360ed6c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:50.810107 18765 raft_consensus.cc:1261] T e1734f079c4744ff876530006eebca64 P d2e72a180c5e43a5ba0728584cc25be4 [term 1 FOLLOWER]: Refusing update from remote peer 32e63505878c446594d70821360ed6c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:50.810545 19438 consensus_queue.cc:1035] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "82b986efd42f463ba147b7067465ac5e" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 40403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:50.810738 19444 consensus_queue.cc:1035] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d2e72a180c5e43a5ba0728584cc25be4" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 36481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:50.812892 19397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.812632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.129:56949)
0811 05:13:50.812743 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.812811 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 05:13:50.813225 19396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.812975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.130:50655)
0811 05:13:50.813088 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.813152 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 05:13:50.815533 19453 mvcc.cc:205] Tried to move back new op lower bound from 7317197745393131520 to 7317197745266528256. Current Snapshot: MvccSnapshot[applied={T|T < 7317197745393131520}]
I20260811 05:13:50.816205 19398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.815747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.131:57353)
0811 05:13:50.816063 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.816122 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245}
I20260811 05:13:50.818574 19397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.818269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.129:49273)
0811 05:13:50.818435 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.818501 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 05:13:50.820375 18288 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:50.820569 18288 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44320:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:50.822742 19396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.822425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.130:51273)
0811 05:13:50.822595 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.822667 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 05:13:50.823459 19398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.821383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45385 (local address 127.7.67.131:37703)
0811 05:13:50.823331 (+ 1948us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.823390 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:45385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1878}
I20260811 05:13:50.823889 19451 mvcc.cc:205] Tried to move back new op lower bound from 7317197745393131520 to 7317197745266528256. Current Snapshot: MvccSnapshot[applied={T|T < 7317197745393131520 or (T in {7317197745393131520})}]
I20260811 05:13:50.825820 18288 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ba069f98f23c4f3b9125de6627e8992d: Sending DeleteTablet for 3 replicas of tablet e1734f079c4744ff876530006eebca64
I20260811 05:13:50.826668 18964 tablet_service.cc:1514] Processing DeleteTablet for tablet e1734f079c4744ff876530006eebca64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:50 UTC) from {username='oryx', principal='oryx/127.7.67.189@KRBTEST.COM'} at 127.0.0.1:40984
I20260811 05:13:50.826694 18963 tablet_service.cc:1514] Processing DeleteTablet for tablet e1734f079c4744ff876530006eebca64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:50 UTC) from {username='oryx', principal='oryx/127.7.67.189@KRBTEST.COM'} at 127.0.0.1:40984
I20260811 05:13:50.826691 19182 tablet_service.cc:1514] Processing DeleteTablet for tablet e1734f079c4744ff876530006eebca64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:50 UTC) from {username='oryx', principal='oryx/127.7.67.189@KRBTEST.COM'} at 127.0.0.1:49844
I20260811 05:13:50.826884 19181 tablet_service.cc:1514] Processing DeleteTablet for tablet e1734f079c4744ff876530006eebca64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:50 UTC) from {username='oryx', principal='oryx/127.7.67.189@KRBTEST.COM'} at 127.0.0.1:49844
I20260811 05:13:50.827076 18734 tablet_service.cc:1514] Processing DeleteTablet for tablet e1734f079c4744ff876530006eebca64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:50 UTC) from {username='oryx', principal='oryx/127.7.67.189@KRBTEST.COM'} at 127.0.0.1:52498
I20260811 05:13:50.827250 19471 tablet_replica.cc:330] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5: stopping tablet replica
I20260811 05:13:50.827432 19474 tablet_replica.cc:330] T e1734f079c4744ff876530006eebca64 P d2e72a180c5e43a5ba0728584cc25be4: stopping tablet replica
I20260811 05:13:50.827564 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 18500
I20260811 05:13:50.827378 19471 raft_consensus.cc:2229] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:13:50.828939 19471 raft_consensus.cc:2258] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:50.829252 19470 tablet_replica.cc:330] T e1734f079c4744ff876530006eebca64 P 82b986efd42f463ba147b7067465ac5e: stopping tablet replica
I20260811 05:13:50.829412 19470 raft_consensus.cc:2229] T e1734f079c4744ff876530006eebca64 P 82b986efd42f463ba147b7067465ac5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:50.829571 19470 raft_consensus.cc:2258] T e1734f079c4744ff876530006eebca64 P 82b986efd42f463ba147b7067465ac5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:50.833537 19470 ts_tablet_manager.cc:1905] T e1734f079c4744ff876530006eebca64 P 82b986efd42f463ba147b7067465ac5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:50.837885 19471 ts_tablet_manager.cc:1905] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:50.840198 19471 ts_tablet_manager.cc:1918] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:50.840279 19471 log.cc:1196] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/e1734f079c4744ff876530006eebca64
I20260811 05:13:50.840713 19471 ts_tablet_manager.cc:1939] T e1734f079c4744ff876530006eebca64 P 32e63505878c446594d70821360ed6c5: Deleting consensus metadata
I20260811 05:13:50.841522 18267 catalog_manager.cc:4841] TS 32e63505878c446594d70821360ed6c5 (127.7.67.131:43549): tablet e1734f079c4744ff876530006eebca64 (table test-table [id=e2444dc97c074f2aa1ab42e8ec06cab3]) successfully deleted
W20260811 05:13:50.841922 18267 catalog_manager.cc:4808] TS 32e63505878c446594d70821360ed6c5 (127.7.67.131:43549): delete failed for tablet e1734f079c4744ff876530006eebca64 because the tablet was not found. No further retry: Not found: Tablet not found: e1734f079c4744ff876530006eebca64
I20260811 05:13:50.842569 19470 ts_tablet_manager.cc:1918] T e1734f079c4744ff876530006eebca64 P 82b986efd42f463ba147b7067465ac5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:50.842643 19470 log.cc:1196] T e1734f079c4744ff876530006eebca64 P 82b986efd42f463ba147b7067465ac5e: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/e1734f079c4744ff876530006eebca64
I20260811 05:13:50.842950 19470 ts_tablet_manager.cc:1939] T e1734f079c4744ff876530006eebca64 P 82b986efd42f463ba147b7067465ac5e: Deleting consensus metadata
I20260811 05:13:50.843693 18265 catalog_manager.cc:4841] TS 82b986efd42f463ba147b7067465ac5e (127.7.67.130:40403): tablet e1734f079c4744ff876530006eebca64 (table test-table [id=e2444dc97c074f2aa1ab42e8ec06cab3]) successfully deleted
W20260811 05:13:50.843904 18265 catalog_manager.cc:4808] TS 82b986efd42f463ba147b7067465ac5e (127.7.67.130:40403): delete failed for tablet e1734f079c4744ff876530006eebca64 because the tablet was not found. No further retry: Not found: Tablet not found: e1734f079c4744ff876530006eebca64
I20260811 05:13:50.846714 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 18866
I20260811 05:13:50.846838 19424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:50.846536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:36481 (local address 127.0.0.1:52510)
0811 05:13:50.846691 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:50.846738 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.129:36481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260811 05:13:50.852392 18264 catalog_manager.cc:4584] TS d2e72a180c5e43a5ba0728584cc25be4 (127.7.67.129:36481): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e1734f079c4744ff876530006eebca64: Network error: Client connection negotiation failed: client connection to 127.7.67.129:36481: connect: Connection refused (error 111)
I20260811 05:13:50.862612 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 19078
I20260811 05:13:50.878981 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 18070
I20260811 05:13:50.894371 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 18237
2026-08-11T05:13:50Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7403 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:13:50 dist-test-slave-04bb krb5kdc[19477](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:13:50 dist-test-slave-04bb krb5kdc[19477](info): set up 2 sockets
Aug 11 05:13:50 dist-test-slave-04bb krb5kdc[19477](info): commencing operation
krb5kdc: starting...
W20260811 05:13:52.167532 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.240s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:13:52 dist-test-slave-04bb krb5kdc[19477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:52Z Disabled control of system clock
WARNING: no policy specified for oryx/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.190.keytab.
Entry for principal oryx/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.190.keytab.
I20260811 05:13:52.299378 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:40745
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.190.keytab
--principal=oryx/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:45583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:40745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:52.414278 19498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:52.414547 19498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:52.414588 19498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:52.418589 19498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:52.418640 19498 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 05:13:52.418658 19498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:52.418675 19498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:52.418691 19498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:52.418706 19498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:52.423928 19498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:45583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:40745
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.190.keytab
--principal=oryx/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:40745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:52.425014 19498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:52.425984 19498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:13:52.434898 19498 server_base.cc:1034] running on GCE node
W20260811 05:13:52.434842 19511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:52.434916 19508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:52.435112 19507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:52.435698 19498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:52.436832 19498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:52.438030 19498 hybrid_clock.cc:648] HybridClock initialized: now 1786425232438015 us; error 27 us; skew 500 ppm
Aug 11 05:13:52 dist-test-slave-04bb krb5kdc[19477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425232, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:52.440821 19498 init.cc:377] Logged in from keytab as oryx/127.7.67.190@KRBTEST.COM (short username oryx)
I20260811 05:13:52.441727 19498 webserver.cc:460] Webserver started at http://127.7.67.190:33731/ using document root <none> and password file <none>
I20260811 05:13:52.442297 19498 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:52.442391 19498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:52.442627 19498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:52.444764 19498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "c0c901ac6d7f4906a97d099e134ffe09"
format_stamp: "Formatted at 2026-08-11 05:13:52 on dist-test-slave-04bb"
server_key: "8ef2ff8b8752239a2f501f52be915b40"
server_key_iv: "07d4b9ee85132230daa656c8a4e746ff"
server_key_version: "encryptionkey@0"
I20260811 05:13:52.445339 19498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "c0c901ac6d7f4906a97d099e134ffe09"
format_stamp: "Formatted at 2026-08-11 05:13:52 on dist-test-slave-04bb"
server_key: "8ef2ff8b8752239a2f501f52be915b40"
server_key_iv: "07d4b9ee85132230daa656c8a4e746ff"
server_key_version: "encryptionkey@0"
I20260811 05:13:52.449072 19498 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:13:52.451501 19522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:52.452757 19498 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:13:52.452903 19498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "c0c901ac6d7f4906a97d099e134ffe09"
format_stamp: "Formatted at 2026-08-11 05:13:52 on dist-test-slave-04bb"
server_key: "8ef2ff8b8752239a2f501f52be915b40"
server_key_iv: "07d4b9ee85132230daa656c8a4e746ff"
server_key_version: "encryptionkey@0"
I20260811 05:13:52.453013 19498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:52.466185 19498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:52.469319 19498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:52.469528 19498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:52.478157 19498 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:40745
I20260811 05:13:52.478178 19618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:40745 every 8 connection(s)
I20260811 05:13:52.479146 19498 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:52.482030 19619 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:52.486090 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 19498
I20260811 05:13:52.486241 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:13:52.486621 7438 external_mini_cluster.cc:1428] Setting key a4d8d5a1ad7809b0057a357894bb716a
I20260811 05:13:52.488358 19619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09: Bootstrap starting.
I20260811 05:13:52.491217 19619 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:52.492132 19619 log.cc:824] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:52.494351 19619 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09: No bootstrap required, opened a new log
Aug 11 05:13:52 dist-test-slave-04bb krb5kdc[19477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425232, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM
I20260811 05:13:52.498667 19619 raft_consensus.cc:348] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0c901ac6d7f4906a97d099e134ffe09" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 40745 } }
I20260811 05:13:52.498904 19619 raft_consensus.cc:374] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:52.498965 19619 raft_consensus.cc:729] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0c901ac6d7f4906a97d099e134ffe09, State: Initialized, Role: FOLLOWER
I20260811 05:13:52.499490 19619 consensus_queue.cc:260] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0c901ac6d7f4906a97d099e134ffe09" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 40745 } }
I20260811 05:13:52.499687 19619 raft_consensus.cc:388] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:52.499759 19619 raft_consensus.cc:482] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:52.499889 19619 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:52.500849 19619 raft_consensus.cc:504] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0c901ac6d7f4906a97d099e134ffe09" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 40745 } }
I20260811 05:13:52.501201 19619 leader_election.cc:302] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c0c901ac6d7f4906a97d099e134ffe09; no voters:
I20260811 05:13:52.501554 19619 leader_election.cc:288] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:52.501646 19625 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:52.501822 19625 raft_consensus.cc:686] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09 [term 1 LEADER]: Becoming Leader. State: Replica: c0c901ac6d7f4906a97d099e134ffe09, State: Running, Role: LEADER
I20260811 05:13:52.502205 19625 consensus_queue.cc:237] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0c901ac6d7f4906a97d099e134ffe09" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 40745 } }
I20260811 05:13:52.502547 19623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:52.488166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52126 (local address 127.7.67.190:40745)
0811 05:13:52.488634 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:52.488657 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:52.488700 (+ 43us) server_negotiation.cc:407] Connection header received
0811 05:13:52.489547 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:52.489578 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:52.490018 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:52.490397 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:52.491113 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:52.492649 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:52.493601 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:52.494260 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:52.494442 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:52.497247 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:52.497253 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:52.497266 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:52.497293 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:52.499116 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:52.499600 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:52.499605 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:52.499610 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:52.499786 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:52.500161 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:52.500164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:52.500166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:52.500664 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:52.500889 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:52.501189 (+ 300us) server_negotiation.cc:299] Negotiation successful
0811 05:13:52.501535 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":111,"threads_started":1}
I20260811 05:13:52.502874 19619 sys_catalog.cc:564] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:52.503876 19626 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c0c901ac6d7f4906a97d099e134ffe09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0c901ac6d7f4906a97d099e134ffe09" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 40745 } } }
I20260811 05:13:52.504017 19626 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:52.504230 19626 sys_catalog.cc:455] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c0c901ac6d7f4906a97d099e134ffe09. Latest consensus state: current_term: 1 leader_uuid: "c0c901ac6d7f4906a97d099e134ffe09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c0c901ac6d7f4906a97d099e134ffe09" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 40745 } } }
I20260811 05:13:52.504329 19626 sys_catalog.cc:458] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09 [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:52.505165 19635 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:52.508380 19635 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:52.513958 19635 catalog_manager.cc:1347] Generated new cluster ID: 211d47c0f42646e69c959e1cdca1798e
I20260811 05:13:52.514037 19635 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:52.532205 19635 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:52.533246 19635 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:52.547960 19635 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09: Generated new TSK 0
I20260811 05:13:52.548712 19635 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.7.67.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.129.keytab.
Entry for principal oryx/127.7.67.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.129.keytab.
I20260811 05:13:52.583806 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:40745
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.129.keytab
--principal=oryx/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:45583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:52.727659 19659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:52.727931 19659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:52.727977 19659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:52.731932 19659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:52.732000 19659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:52.732038 19659 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:13:52.732110 19659 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:13:52.737258 19659 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:45583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.129.keytab
--principal=oryx/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.67.190:40745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:52.738282 19659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:52.739324 19659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:52.748924 19670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:52.748943 19675 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:52.748992 19668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:52.749383 19659 server_base.cc:1034] running on GCE node
I20260811 05:13:52.749769 19659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:52.750415 19659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:52.751600 19659 hybrid_clock.cc:648] HybridClock initialized: now 1786425232751583 us; error 37 us; skew 500 ppm
Aug 11 05:13:52 dist-test-slave-04bb krb5kdc[19477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425232, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:52.754370 19659 init.cc:377] Logged in from keytab as oryx/127.7.67.129@KRBTEST.COM (short username oryx)
I20260811 05:13:52.755210 19659 webserver.cc:460] Webserver started at http://127.7.67.129:42355/ using document root <none> and password file <none>
I20260811 05:13:52.755867 19659 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:52.755961 19659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:52.756228 19659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:52.758390 19659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "945234845f614c6793619eee11183edc"
format_stamp: "Formatted at 2026-08-11 05:13:52 on dist-test-slave-04bb"
server_key: "d3cc5e482364b8d488d4b6ed90275b0a"
server_key_iv: "53235afc4cbee58576aba963f6aa51ba"
server_key_version: "encryptionkey@0"
I20260811 05:13:52.758970 19659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "945234845f614c6793619eee11183edc"
format_stamp: "Formatted at 2026-08-11 05:13:52 on dist-test-slave-04bb"
server_key: "d3cc5e482364b8d488d4b6ed90275b0a"
server_key_iv: "53235afc4cbee58576aba963f6aa51ba"
server_key_version: "encryptionkey@0"
I20260811 05:13:52.762820 19659 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 05:13:52.765251 19688 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:52.766467 19659 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:13:52.766615 19659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "945234845f614c6793619eee11183edc"
format_stamp: "Formatted at 2026-08-11 05:13:52 on dist-test-slave-04bb"
server_key: "d3cc5e482364b8d488d4b6ed90275b0a"
server_key_iv: "53235afc4cbee58576aba963f6aa51ba"
server_key_version: "encryptionkey@0"
I20260811 05:13:52.766741 19659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:52.775921 19659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:52.778597 19659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:52.778802 19659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:52.780146 19659 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:52.780227 19659 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:52.780299 19659 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:52.780345 19659 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 05:13:52 dist-test-slave-04bb krb5kdc[19477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425232, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM
I20260811 05:13:52.792560 19659 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:46641
I20260811 05:13:52.792666 19875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:46641 every 8 connection(s)
I20260811 05:13:52.793449 19659 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:52.795076 19623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:52.781024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:47155 (local address 127.7.67.190:40745)
0811 05:13:52.781178 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:52.781185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:52.782084 (+ 899us) server_negotiation.cc:407] Connection header received
0811 05:13:52.783400 (+ 1316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:52.783406 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:52.783477 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:52.783542 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:52.785416 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:52.786235 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:52.787669 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:52.788091 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:52.788206 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:52.791398 (+ 3192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:52.791403 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:52.791405 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:52.791437 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:52.793076 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:52.793583 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:52.793590 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:52.793592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:52.793676 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:52.794096 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:52.794100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:52.794104 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:52.794353 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:52.794433 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:52.794858 (+ 425us) server_negotiation.cc:299] Negotiation successful
0811 05:13:52.794956 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 05:13:52.795948 19709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:52.781405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:40745 (local address 127.7.67.129:47155)
0811 05:13:52.781902 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:52.781960 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:52.783115 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:52.783994 (+ 879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:52.784003 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:52.784496 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:52.785231 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:52.785256 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:52.786371 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:52.786375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:52.787535 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:52.787550 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:52.788277 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:52.788285 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:52.788506 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:52.789189 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:52.789205 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:52.791214 (+ 2009us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:52.793195 (+ 1981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:52.793201 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:52.793215 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:52.793476 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:52.793772 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:52.793775 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:52.793777 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:52.793977 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:52.794431 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:52.794436 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:52.794643 (+ 207us) client_negotiation.cc:769] Sending connection context
0811 05:13:52.794805 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 05:13:52.795039 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":105,"threads_started":1}
I20260811 05:13:52.797268 19877 heartbeater.cc:347] Connected to a master server at 127.7.67.190:40745
I20260811 05:13:52.797434 19877 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:52.797917 19877 heartbeater.cc:511] Master 127.7.67.190:40745 requested a full tablet report, sending...
I20260811 05:13:52.799424 19547 ts_manager.cc:194] Registered new tserver with Master: 945234845f614c6793619eee11183edc (127.7.67.129:46641)
I20260811 05:13:52.800653 19547 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.67.129@KRBTEST.COM'} at 127.7.67.129:47155
I20260811 05:13:52.801525 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 19659
I20260811 05:13:52.801656 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:13:52.801941 7438 external_mini_cluster.cc:1428] Setting key f9e67462094e92fea2fe9cc7ba0d7120
I20260811 05:13:52.807247 19623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:52.799486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:47661 (local address 127.7.67.190:40745)
0811 05:13:52.799604 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:52.799608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:52.799777 (+ 169us) server_negotiation.cc:407] Connection header received
0811 05:13:52.799911 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:52.799914 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:52.799961 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:52.800045 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:52.800473 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:52.801086 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:52.802211 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:52.802517 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:52.802622 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:52.803828 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:52.803834 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:52.803840 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:52.803870 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:52.805568 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:52.806127 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:52.806130 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:52.806131 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:52.806205 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:52.806511 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:52.806514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:52.806515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:52.806667 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:52.806759 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:52.807050 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 05:13:52.807138 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 05:13:52.807389 19894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:52.799371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:40745 (local address 127.7.67.129:47661)
0811 05:13:52.799696 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:52.799707 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:52.799824 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:52.800061 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:52.800064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:52.800266 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:52.800358 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:52.800363 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:52.801214 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:52.801217 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:52.802077 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:52.802088 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:52.802640 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:52.802644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:52.802733 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:52.803292 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:52.803307 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:52.803700 (+ 393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:52.805718 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:52.805724 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:52.805728 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:52.806024 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:52.806306 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:52.806311 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:52.806315 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:52.806417 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:52.806760 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:52.806765 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:52.806885 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 05:13:52.807001 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 05:13:52.807164 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":90,"threads_started":1}
WARNING: no policy specified for oryx/127.7.67.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.130.keytab.
Entry for principal oryx/127.7.67.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.130.keytab.
I20260811 05:13:52.831931 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:40745
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.130.keytab
--principal=oryx/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:45583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:52.949502 19897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:52.949738 19897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:52.949780 19897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:52.953686 19897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:52.953739 19897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:52.953778 19897 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:13:52.953810 19897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:13:52.959139 19897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:45583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.130.keytab
--principal=oryx/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.67.190:40745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:52.960129 19897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:52.961210 19897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:52.971294 19908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:52.971283 19907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:52.971334 19913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:52.971481 19897 server_base.cc:1034] running on GCE node
I20260811 05:13:52.971969 19897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:52.972664 19897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:52.973834 19897 hybrid_clock.cc:648] HybridClock initialized: now 1786425232973803 us; error 44 us; skew 500 ppm
Aug 11 05:13:52 dist-test-slave-04bb krb5kdc[19477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425232, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:52.976755 19897 init.cc:377] Logged in from keytab as oryx/127.7.67.130@KRBTEST.COM (short username oryx)
I20260811 05:13:52.977644 19897 webserver.cc:460] Webserver started at http://127.7.67.130:43993/ using document root <none> and password file <none>
I20260811 05:13:52.978273 19897 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:52.978416 19897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:52.978703 19897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:52.981117 19897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "2906ab24d87b4e97abae1f364a25e903"
format_stamp: "Formatted at 2026-08-11 05:13:52 on dist-test-slave-04bb"
server_key: "6781adfd5a9468ff72bb1f6dd953fe5c"
server_key_iv: "31d32018f6714d6a8758966323d597fb"
server_key_version: "encryptionkey@0"
I20260811 05:13:52.981716 19897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "2906ab24d87b4e97abae1f364a25e903"
format_stamp: "Formatted at 2026-08-11 05:13:52 on dist-test-slave-04bb"
server_key: "6781adfd5a9468ff72bb1f6dd953fe5c"
server_key_iv: "31d32018f6714d6a8758966323d597fb"
server_key_version: "encryptionkey@0"
I20260811 05:13:52.985621 19897 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 05:13:52.988157 19927 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:52.989508 19897 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:13:52.989677 19897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "2906ab24d87b4e97abae1f364a25e903"
format_stamp: "Formatted at 2026-08-11 05:13:52 on dist-test-slave-04bb"
server_key: "6781adfd5a9468ff72bb1f6dd953fe5c"
server_key_iv: "31d32018f6714d6a8758966323d597fb"
server_key_version: "encryptionkey@0"
I20260811 05:13:52.989802 19897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:53.021310 19897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:53.024652 19897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:53.024860 19897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:53.026450 19897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:53.026523 19897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:53.026595 19897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:53.026639 19897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 05:13:53 dist-test-slave-04bb krb5kdc[19477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425232, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM
I20260811 05:13:53.037983 19897 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:42179
I20260811 05:13:53.038041 20108 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:42179 every 8 connection(s)
I20260811 05:13:53.038882 19897 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:53.039429 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 19897
I20260811 05:13:53.039549 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:13:53.039889 7438 external_mini_cluster.cc:1428] Setting key 4dab87d770be42d558913547f379d476
I20260811 05:13:53.041600 19623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.027402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:37867 (local address 127.7.67.190:40745)
0811 05:13:53.027545 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:53.027549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:53.028718 (+ 1169us) server_negotiation.cc:407] Connection header received
0811 05:13:53.029873 (+ 1155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:53.029877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:53.029951 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:53.030065 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:53.031543 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.032101 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.033487 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.033716 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.033841 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.037254 (+ 3413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:53.037258 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:53.037261 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:53.037305 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:53.038902 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:53.039534 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:53.039538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:53.039540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:53.039635 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:53.040178 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:53.040182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:53.040184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:53.040495 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:53.040656 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:53.041292 (+ 636us) server_negotiation.cc:299] Negotiation successful
0811 05:13:53.041444 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:13:53.042353 19942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.027982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:40745 (local address 127.7.67.130:37867)
0811 05:13:53.028497 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:53.028600 (+ 103us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:53.029558 (+ 958us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:53.030272 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:53.030299 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:53.030800 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:53.031345 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.031383 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.032273 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.032288 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.033348 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.033358 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.033895 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.033899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.034110 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.034736 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:53.034757 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:53.037083 (+ 2326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:53.039020 (+ 1937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:53.039028 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:53.039041 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:53.039400 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:53.039751 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:53.039755 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:53.039757 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:53.039956 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:53.040677 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:53.040683 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:53.040971 (+ 288us) client_negotiation.cc:769] Sending connection context
0811 05:13:53.041158 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 05:13:53.041484 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":154,"threads_started":1}
I20260811 05:13:53.043622 20109 heartbeater.cc:347] Connected to a master server at 127.7.67.190:40745
I20260811 05:13:53.043825 20109 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:53.044353 20109 heartbeater.cc:511] Master 127.7.67.190:40745 requested a full tablet report, sending...
I20260811 05:13:53.045487 19547 ts_manager.cc:194] Registered new tserver with Master: 2906ab24d87b4e97abae1f364a25e903 (127.7.67.130:42179)
I20260811 05:13:53.046352 19547 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.67.130@KRBTEST.COM'} at 127.7.67.130:37867
WARNING: no policy specified for oryx/127.7.67.131@KRBTEST.COM; defaulting to no policy
I20260811 05:13:53.053085 20121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.046003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:40745 (local address 127.7.67.130:59101)
0811 05:13:53.046313 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:53.046324 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:53.046434 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:53.046761 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:53.046764 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:53.046988 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:53.047094 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.047100 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.047927 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.047930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.048575 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.048591 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.049049 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.049053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.049153 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.049643 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:53.049654 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:53.049960 (+ 306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:53.051626 (+ 1666us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:53.051632 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:53.051634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:53.051846 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:53.052131 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:53.052134 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:53.052136 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:53.052221 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:53.052676 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:53.052679 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:53.052756 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 05:13:53.052850 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 05:13:53.052950 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":92,"threads_started":1}
I20260811 05:13:53.053115 19623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.046050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:59101 (local address 127.7.67.190:40745)
0811 05:13:53.046161 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:53.046166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:53.046397 (+ 231us) server_negotiation.cc:407] Connection header received
0811 05:13:53.046509 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:53.046512 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:53.046559 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:53.046648 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:53.047210 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.047749 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.048702 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.048933 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.049036 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.050097 (+ 1061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:53.050100 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:53.050103 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:53.050126 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:53.051514 (+ 1388us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:53.051949 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:53.051953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:53.051955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:53.052035 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:53.052323 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:53.052326 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:53.052328 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:53.052579 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:53.052707 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:53.052918 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 05:13:53.053010 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.131.keytab.
Entry for principal oryx/127.7.67.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.131.keytab.
I20260811 05:13:53.071731 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:40745
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.131.keytab
--principal=oryx/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:45583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:53.190544 20125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:53.190789 20125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:53.190832 20125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:53.195010 20125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:53.195073 20125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:53.195149 20125 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 05:13:53.195209 20125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:13:53.200670 20125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:45583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.131.keytab
--principal=oryx/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.7.67.190:40745
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:53.201737 20125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:53.202878 20125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:53.212914 20134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:53.212960 20132 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:53.212939 20140 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:53.213155 20125 server_base.cc:1034] running on GCE node
I20260811 05:13:53.213685 20125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:53.214298 20125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:53.215466 20125 hybrid_clock.cc:648] HybridClock initialized: now 1786425233215444 us; error 33 us; skew 500 ppm
Aug 11 05:13:53 dist-test-slave-04bb krb5kdc[19477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425233, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:53.218458 20125 init.cc:377] Logged in from keytab as oryx/127.7.67.131@KRBTEST.COM (short username oryx)
I20260811 05:13:53.219472 20125 webserver.cc:460] Webserver started at http://127.7.67.131:35527/ using document root <none> and password file <none>
I20260811 05:13:53.220173 20125 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:53.220280 20125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:53.220572 20125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:53.222743 20125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "73460f11a7024fc9843372b07ee57dae"
format_stamp: "Formatted at 2026-08-11 05:13:53 on dist-test-slave-04bb"
server_key: "5a6ae3386789529aa1fcd602efe12158"
server_key_iv: "5a475c52f8b567f7711bdc4113656957"
server_key_version: "encryptionkey@0"
I20260811 05:13:53.223357 20125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "73460f11a7024fc9843372b07ee57dae"
format_stamp: "Formatted at 2026-08-11 05:13:53 on dist-test-slave-04bb"
server_key: "5a6ae3386789529aa1fcd602efe12158"
server_key_iv: "5a475c52f8b567f7711bdc4113656957"
server_key_version: "encryptionkey@0"
I20260811 05:13:53.227350 20125 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 05:13:53.230011 20153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:53.231349 20125 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:53.231510 20125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "73460f11a7024fc9843372b07ee57dae"
format_stamp: "Formatted at 2026-08-11 05:13:53 on dist-test-slave-04bb"
server_key: "5a6ae3386789529aa1fcd602efe12158"
server_key_iv: "5a475c52f8b567f7711bdc4113656957"
server_key_version: "encryptionkey@0"
I20260811 05:13:53.231642 20125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:53.260674 20125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:53.264004 20125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:53.264219 20125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:53.265749 20125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:53.265825 20125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:53.265901 20125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:53.265945 20125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 05:13:53 dist-test-slave-04bb krb5kdc[19477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425233, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM
I20260811 05:13:53.277292 20125 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:41747
I20260811 05:13:53.277650 20352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:41747 every 8 connection(s)
I20260811 05:13:53.278244 20125 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:13:53.279299 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 20125
I20260811 05:13:53.279443 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:13:53.279783 7438 external_mini_cluster.cc:1428] Setting key 7040c9124da378b08bd6fc28c5cb0b72
I20260811 05:13:53.280400 19623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.266626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:57481 (local address 127.7.67.190:40745)
0811 05:13:53.266852 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:53.266859 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:53.267657 (+ 798us) server_negotiation.cc:407] Connection header received
0811 05:13:53.268817 (+ 1160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:53.268821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:53.268886 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:53.268951 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:53.270603 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.271532 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.273481 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.273798 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.273927 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.276391 (+ 2464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:53.276395 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:53.276398 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:53.276430 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:53.278118 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:53.278697 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:53.278702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:53.278715 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:53.278812 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:53.279272 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:53.279277 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:53.279279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:53.279524 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:53.279634 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:53.280170 (+ 536us) server_negotiation.cc:299] Negotiation successful
0811 05:13:53.280279 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 05:13:53.281364 20168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.267010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:40745 (local address 127.7.67.131:57481)
0811 05:13:53.267465 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:53.267517 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:53.268553 (+ 1036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:53.269426 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:53.269433 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:53.269752 (+ 319us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:53.270322 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.270344 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.271719 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.271724 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.273335 (+ 1611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.273351 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.273981 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.273985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.274162 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.274768 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:53.274784 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:53.276213 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:53.278249 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:53.278255 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:53.278270 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:53.278580 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:53.278933 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:53.278952 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:53.278955 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:53.279131 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:53.279643 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:53.279648 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:53.279914 (+ 266us) client_negotiation.cc:769] Sending connection context
0811 05:13:53.280089 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 05:13:53.280444 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":132,"threads_started":1}
I20260811 05:13:53.282718 20354 heartbeater.cc:347] Connected to a master server at 127.7.67.190:40745
I20260811 05:13:53.282895 20354 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:53.283377 20354 heartbeater.cc:511] Master 127.7.67.190:40745 requested a full tablet report, sending...
I20260811 05:13:53.284425 19547 ts_manager.cc:194] Registered new tserver with Master: 73460f11a7024fc9843372b07ee57dae (127.7.67.131:41747)
I20260811 05:13:53.285035 19547 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.7.67.131@KRBTEST.COM'} at 127.7.67.131:57481
I20260811 05:13:53.292239 19623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.285177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:57151 (local address 127.7.67.190:40745)
0811 05:13:53.285318 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:53.285324 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:53.285663 (+ 339us) server_negotiation.cc:407] Connection header received
0811 05:13:53.285786 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:53.285790 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:53.285829 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:53.285917 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:53.286346 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.286844 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.287661 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.287907 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.288017 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.289109 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:53.289112 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:53.289115 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:53.289138 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:53.290627 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:53.291049 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:53.291052 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:53.291054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:53.291133 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:53.291420 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:53.291424 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:53.291425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:53.291592 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:53.291705 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:53.291971 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 05:13:53.292076 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 05:13:53.292248 20362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.285120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:40745 (local address 127.7.67.131:57151)
0811 05:13:53.285565 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:53.285578 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:53.285702 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:53.285932 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:53.285935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:53.286135 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:53.286231 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.286237 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.286974 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.286978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.287541 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.287550 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.288032 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.288035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.288137 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.288655 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:53.288667 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:53.288972 (+ 305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:53.290740 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:53.290746 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:53.290748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:53.290943 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:53.291232 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:53.291236 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:53.291238 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:53.291326 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:53.291694 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:53.291698 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:53.291796 (+ 98us) client_negotiation.cc:769] Sending connection context
0811 05:13:53.291899 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 05:13:53.292039 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":278,"threads_started":1}
I20260811 05:13:53.293921 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 05:13:53 dist-test-slave-04bb krb5kdc[19477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425233, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 05:13:53 dist-test-slave-04bb krb5kdc[19477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:53 dist-test-slave-04bb krb5kdc[19477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 05:13:53.319460 19623 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:53.313500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52132 (local address 127.7.67.190:40745)
0811 05:13:53.313633 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:53.313637 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:53.313650 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:13:53.313702 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:53.313705 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:53.313742 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:53.313823 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:53.314471 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.315003 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.315830 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.316041 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.316122 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.319273 (+ 3151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52132: BlockingRecv error: recv got EOF from 127.0.0.1:52132 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 05:13:53 dist-test-slave-04bb krb5kdc[19477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425233, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM
I20260811 05:13:53.328447 19623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.320863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52144 (local address 127.7.67.190:40745)
0811 05:13:53.320995 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:53.320998 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:53.321061 (+ 63us) server_negotiation.cc:407] Connection header received
0811 05:13:53.321148 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:53.321150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:53.321185 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:53.321247 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:53.321771 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.322247 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.323013 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.323204 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.323280 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.325549 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:53.325552 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:53.325554 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:53.325577 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:53.326994 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:53.327355 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:53.327358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:53.327359 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:53.327430 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:53.327659 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:53.327661 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:53.327662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:53.327826 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:53.327940 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:53.328171 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 05:13:53.328286 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 05:13:53.331246 19547 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:13:53.334282 19547 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:53.349277 20392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.342994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:42179 (local address 127.0.0.1:45470)
0811 05:13:53.344323 (+ 1329us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:53.344338 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:53.344438 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:53.344919 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:53.344925 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:53.344947 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:53.345031 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.345039 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.346435 (+ 1396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.346438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.347487 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.347498 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.348930 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.348936 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.349022 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.349042 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 05:13:53.349108 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 05:13:53.349162 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1269,"thread_start_us":128,"threads_started":1}
I20260811 05:13:53.349311 20399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.343501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48188 (local address 127.7.67.129:46641)
0811 05:13:53.344141 (+ 640us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:53.344147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:53.344298 (+ 151us) server_negotiation.cc:407] Connection header received
0811 05:13:53.344481 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:53.344488 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:53.344771 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:53.344934 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:53.345358 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.346404 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.347983 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.348514 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.348690 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.348845 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:53.349003 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 05:13:53.349116 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":547,"thread_start_us":92,"threads_started":1}
I20260811 05:13:53.349601 20396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.342974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57300 (local address 127.7.67.131:41747)
0811 05:13:53.344049 (+ 1075us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:53.344064 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:53.344090 (+ 26us) server_negotiation.cc:407] Connection header received
0811 05:13:53.344173 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:53.344181 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:53.344455 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:53.344656 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:53.344963 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.346088 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.347770 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.348614 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.348726 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.348885 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:53.349337 (+ 452us) server_negotiation.cc:299] Negotiation successful
0811 05:13:53.349450 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":972,"thread_start_us":116,"threads_started":1}
I20260811 05:13:53.350099 20400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.343524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45470 (local address 127.7.67.130:42179)
0811 05:13:53.344167 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:53.344183 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:53.344414 (+ 231us) server_negotiation.cc:407] Connection header received
0811 05:13:53.344567 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:53.344574 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:53.344766 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:53.344934 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:53.345127 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.346082 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.347628 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.348585 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.348712 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.348885 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:53.349861 (+ 976us) server_negotiation.cc:299] Negotiation successful
0811 05:13:53.349958 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":179,"threads_started":1}
I20260811 05:13:53.349241 20397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.342676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:46641 (local address 127.0.0.1:48188)
0811 05:13:53.344175 (+ 1499us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:53.344187 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:53.344311 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:53.345086 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:53.345092 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:53.345114 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:53.345220 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.345227 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.346738 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.346751 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.347861 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.347880 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.348689 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.348693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.348810 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.348846 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 05:13:53.348961 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 05:13:53.349087 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1419,"mutex_wait_us":22,"thread_start_us":108,"threads_started":1}
I20260811 05:13:53.349463 20395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.342665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:41747 (local address 127.0.0.1:57300)
0811 05:13:53.343716 (+ 1051us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:53.343774 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:53.343916 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:53.344647 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:53.344653 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:53.344674 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:53.344815 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.344822 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.346222 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.346226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.347641 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.347653 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.348963 (+ 1310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.348967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.349067 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.349092 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 05:13:53.349174 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 05:13:53.349248 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":855,"mutex_wait_us":30,"thread_start_us":72,"threads_started":1}
I20260811 05:13:53.352924 20238 tablet_service.cc:1467] Processing CreateTablet for tablet e34c87fda1ce4e18a8e10cb33464309a (DEFAULT_TABLE table=test-table [id=0ab7b41be7b64376ac1ad71444bd34e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:53.352952 20009 tablet_service.cc:1467] Processing CreateTablet for tablet e34c87fda1ce4e18a8e10cb33464309a (DEFAULT_TABLE table=test-table [id=0ab7b41be7b64376ac1ad71444bd34e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:53.354038 20238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e34c87fda1ce4e18a8e10cb33464309a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:53.354120 20009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e34c87fda1ce4e18a8e10cb33464309a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:53.355155 19764 tablet_service.cc:1467] Processing CreateTablet for tablet e34c87fda1ce4e18a8e10cb33464309a (DEFAULT_TABLE table=test-table [id=0ab7b41be7b64376ac1ad71444bd34e7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:53.356132 19764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e34c87fda1ce4e18a8e10cb33464309a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:53.361949 20404 tablet_bootstrap.cc:492] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae: Bootstrap starting.
I20260811 05:13:53.362470 20403 tablet_bootstrap.cc:492] T e34c87fda1ce4e18a8e10cb33464309a P 2906ab24d87b4e97abae1f364a25e903: Bootstrap starting.
I20260811 05:13:53.363225 20406 tablet_bootstrap.cc:492] T e34c87fda1ce4e18a8e10cb33464309a P 945234845f614c6793619eee11183edc: Bootstrap starting.
I20260811 05:13:53.364791 20403 tablet_bootstrap.cc:654] T e34c87fda1ce4e18a8e10cb33464309a P 2906ab24d87b4e97abae1f364a25e903: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:53.364845 20404 tablet_bootstrap.cc:654] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:53.365468 20406 tablet_bootstrap.cc:654] T e34c87fda1ce4e18a8e10cb33464309a P 945234845f614c6793619eee11183edc: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:53.365531 20403 log.cc:824] T e34c87fda1ce4e18a8e10cb33464309a P 2906ab24d87b4e97abae1f364a25e903: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:53.365815 20404 log.cc:824] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:53.366381 20406 log.cc:824] T e34c87fda1ce4e18a8e10cb33464309a P 945234845f614c6793619eee11183edc: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:53.367535 20403 tablet_bootstrap.cc:492] T e34c87fda1ce4e18a8e10cb33464309a P 2906ab24d87b4e97abae1f364a25e903: No bootstrap required, opened a new log
I20260811 05:13:53.367753 20403 ts_tablet_manager.cc:1397] T e34c87fda1ce4e18a8e10cb33464309a P 2906ab24d87b4e97abae1f364a25e903: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:13:53.367920 20404 tablet_bootstrap.cc:492] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae: No bootstrap required, opened a new log
I20260811 05:13:53.368103 20404 ts_tablet_manager.cc:1397] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:13:53.368268 20406 tablet_bootstrap.cc:492] T e34c87fda1ce4e18a8e10cb33464309a P 945234845f614c6793619eee11183edc: No bootstrap required, opened a new log
I20260811 05:13:53.368409 20406 ts_tablet_manager.cc:1397] T e34c87fda1ce4e18a8e10cb33464309a P 945234845f614c6793619eee11183edc: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:13:53.371809 20403 raft_consensus.cc:348] T e34c87fda1ce4e18a8e10cb33464309a P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } } peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } }
I20260811 05:13:53.372084 20403 raft_consensus.cc:374] T e34c87fda1ce4e18a8e10cb33464309a P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:53.371959 20404 raft_consensus.cc:348] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } } peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } }
I20260811 05:13:53.372169 20404 raft_consensus.cc:374] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:53.372176 20403 raft_consensus.cc:729] T e34c87fda1ce4e18a8e10cb33464309a P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2906ab24d87b4e97abae1f364a25e903, State: Initialized, Role: FOLLOWER
I20260811 05:13:53.372232 20404 raft_consensus.cc:729] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73460f11a7024fc9843372b07ee57dae, State: Initialized, Role: FOLLOWER
I20260811 05:13:53.372674 20403 consensus_queue.cc:260] T e34c87fda1ce4e18a8e10cb33464309a P 2906ab24d87b4e97abae1f364a25e903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } } peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } }
I20260811 05:13:53.372674 20404 consensus_queue.cc:260] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } } peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } }
I20260811 05:13:53.372673 20406 raft_consensus.cc:348] T e34c87fda1ce4e18a8e10cb33464309a P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } } peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } }
I20260811 05:13:53.372949 20406 raft_consensus.cc:374] T e34c87fda1ce4e18a8e10cb33464309a P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:53.373078 20406 raft_consensus.cc:729] T e34c87fda1ce4e18a8e10cb33464309a P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 945234845f614c6793619eee11183edc, State: Initialized, Role: FOLLOWER
I20260811 05:13:53.373567 20406 consensus_queue.cc:260] T e34c87fda1ce4e18a8e10cb33464309a P 945234845f614c6793619eee11183edc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } } peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } }
I20260811 05:13:53.373699 20403 ts_tablet_manager.cc:1428] T e34c87fda1ce4e18a8e10cb33464309a P 2906ab24d87b4e97abae1f364a25e903: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:13:53.373700 20404 ts_tablet_manager.cc:1428] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:13:53.373757 20109 heartbeater.cc:503] Master 127.7.67.190:40745 was elected leader, sending a full tablet report...
I20260811 05:13:53.374266 19877 heartbeater.cc:503] Master 127.7.67.190:40745 was elected leader, sending a full tablet report...
I20260811 05:13:53.374616 20406 ts_tablet_manager.cc:1428] T e34c87fda1ce4e18a8e10cb33464309a P 945234845f614c6793619eee11183edc: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260811 05:13:53.375106 20354 heartbeater.cc:503] Master 127.7.67.190:40745 was elected leader, sending a full tablet report...
I20260811 05:13:53.497699 20416 raft_consensus.cc:482] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:53.497954 20416 raft_consensus.cc:504] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } } peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } }
I20260811 05:13:53.499198 20416 leader_election.cc:288] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 945234845f614c6793619eee11183edc (127.7.67.129:46641), 2906ab24d87b4e97abae1f364a25e903 (127.7.67.130:42179)
I20260811 05:13:53.502841 20168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.499467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:46641 (local address 127.7.67.131:33575)
0811 05:13:53.499591 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:53.499609 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:53.499707 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:53.499991 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:53.499994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:53.500014 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:53.500159 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.500165 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.500929 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.500934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.501704 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.501722 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.502398 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.502402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.502511 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.502539 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 05:13:53.502629 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 05:13:53.502701 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"spinlock_wait_cycles":47744}
I20260811 05:13:53.502841 20399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.499563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:33575 (local address 127.7.67.129:46641)
0811 05:13:53.499749 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:53.499756 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:53.499774 (+ 18us) server_negotiation.cc:407] Connection header received
0811 05:13:53.499839 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:53.499842 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:53.499885 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:53.499973 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:53.500279 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.500817 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.501841 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.502268 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.502398 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.502480 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:53.502645 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 05:13:53.502721 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260811 05:13:53.503664 20419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.499476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:42179 (local address 127.7.67.131:55425)
0811 05:13:53.499912 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:53.499924 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:53.500076 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:53.500351 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:53.500354 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:53.500370 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:53.500937 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.500946 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.501679 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.501685 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.502446 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.502459 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.503260 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.503266 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.503364 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.503383 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 05:13:53.503471 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 05:13:53.503530 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":119,"spinlock_wait_cycles":768,"thread_start_us":135,"threads_started":1}
I20260811 05:13:53.503675 20400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.499577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:55425 (local address 127.7.67.130:42179)
0811 05:13:53.499731 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:53.499752 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:53.500006 (+ 254us) server_negotiation.cc:407] Connection header received
0811 05:13:53.500194 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:53.500197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:53.500246 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:53.500330 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:53.501063 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.501564 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.502580 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.503084 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.503219 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.503283 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:53.503491 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 05:13:53.503571 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 05:13:53.503650 19793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e34c87fda1ce4e18a8e10cb33464309a" candidate_uuid: "73460f11a7024fc9843372b07ee57dae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945234845f614c6793619eee11183edc" is_pre_election: true
I20260811 05:13:53.504040 19793 raft_consensus.cc:2454] T e34c87fda1ce4e18a8e10cb33464309a P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73460f11a7024fc9843372b07ee57dae in term 0.
I20260811 05:13:53.504446 20041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e34c87fda1ce4e18a8e10cb33464309a" candidate_uuid: "73460f11a7024fc9843372b07ee57dae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2906ab24d87b4e97abae1f364a25e903" is_pre_election: true
I20260811 05:13:53.504632 20154 leader_election.cc:302] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73460f11a7024fc9843372b07ee57dae, 945234845f614c6793619eee11183edc; no voters:
I20260811 05:13:53.504755 20041 raft_consensus.cc:2454] T e34c87fda1ce4e18a8e10cb33464309a P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73460f11a7024fc9843372b07ee57dae in term 0.
I20260811 05:13:53.504985 20416 raft_consensus.cc:2781] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:53.505125 20416 raft_consensus.cc:482] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:53.505185 20416 raft_consensus.cc:3037] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:53.506265 20416 raft_consensus.cc:504] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } } peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } }
I20260811 05:13:53.506578 20416 leader_election.cc:288] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae [CANDIDATE]: Term 1 election: Requested vote from peers 945234845f614c6793619eee11183edc (127.7.67.129:46641), 2906ab24d87b4e97abae1f364a25e903 (127.7.67.130:42179)
I20260811 05:13:53.506896 19793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e34c87fda1ce4e18a8e10cb33464309a" candidate_uuid: "73460f11a7024fc9843372b07ee57dae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945234845f614c6793619eee11183edc"
I20260811 05:13:53.506927 20041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e34c87fda1ce4e18a8e10cb33464309a" candidate_uuid: "73460f11a7024fc9843372b07ee57dae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2906ab24d87b4e97abae1f364a25e903"
I20260811 05:13:53.507018 20041 raft_consensus.cc:3037] T e34c87fda1ce4e18a8e10cb33464309a P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:53.507033 19793 raft_consensus.cc:3037] T e34c87fda1ce4e18a8e10cb33464309a P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:53.508062 20041 raft_consensus.cc:2454] T e34c87fda1ce4e18a8e10cb33464309a P 2906ab24d87b4e97abae1f364a25e903 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73460f11a7024fc9843372b07ee57dae in term 1.
I20260811 05:13:53.508152 19793 raft_consensus.cc:2454] T e34c87fda1ce4e18a8e10cb33464309a P 945234845f614c6793619eee11183edc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73460f11a7024fc9843372b07ee57dae in term 1.
I20260811 05:13:53.508404 20157 leader_election.cc:302] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2906ab24d87b4e97abae1f364a25e903, 73460f11a7024fc9843372b07ee57dae; no voters:
I20260811 05:13:53.508577 20416 raft_consensus.cc:2781] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:53.508795 20416 raft_consensus.cc:686] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae [term 1 LEADER]: Becoming Leader. State: Replica: 73460f11a7024fc9843372b07ee57dae, State: Running, Role: LEADER
I20260811 05:13:53.509109 20416 consensus_queue.cc:237] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } } peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } }
I20260811 05:13:53.512511 19547 catalog_manager.cc:5495] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae reported cstate change: term changed from 0 to 1, leader changed from <none> to 73460f11a7024fc9843372b07ee57dae (127.7.67.131). New cstate: current_term: 1 leader_uuid: "73460f11a7024fc9843372b07ee57dae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } health_report { overall_health: UNKNOWN } } }
W20260811 05:13:53.529237 20355 tablet.cc:2369] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:13:53.536476 20396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.532911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57316 (local address 127.7.67.131:41747)
0811 05:13:53.533041 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:53.533046 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:53.533069 (+ 23us) server_negotiation.cc:407] Connection header received
0811 05:13:53.533141 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:53.533143 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:53.533197 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:53.533312 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:53.533574 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.534056 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.534916 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.535182 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.535270 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.535518 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:53.536002 (+ 484us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:53.536092 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:53.536183 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 05:13:53.536295 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 05:13:53.540197 20111 tablet.cc:2369] T e34c87fda1ce4e18a8e10cb33464309a P 2906ab24d87b4e97abae1f364a25e903: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:13:53.542403 19793 raft_consensus.cc:1261] T e34c87fda1ce4e18a8e10cb33464309a P 945234845f614c6793619eee11183edc [term 1 FOLLOWER]: Refusing update from remote peer 73460f11a7024fc9843372b07ee57dae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:53.542658 20041 raft_consensus.cc:1261] T e34c87fda1ce4e18a8e10cb33464309a P 2906ab24d87b4e97abae1f364a25e903 [term 1 FOLLOWER]: Refusing update from remote peer 73460f11a7024fc9843372b07ee57dae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:53.543277 20420 consensus_queue.cc:1035] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae [LEADER]: Connected to new peer: Peer: permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:53.543503 20416 consensus_queue.cc:1035] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae [LEADER]: Connected to new peer: Peer: permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 05:13:53.544883 19885 tablet.cc:2369] T e34c87fda1ce4e18a8e10cb33464309a P 945234845f614c6793619eee11183edc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:13:53.552806 20425 mvcc.cc:205] Tried to move back new op lower bound from 7317197756581552128 to 7317197756456214528. Current Snapshot: MvccSnapshot[applied={T|T < 7317197756581552128}]
I20260811 05:13:53.563105 19548 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:53.563278 19548 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:53.566018 19548 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09: Sending DeleteTablet for 3 replicas of tablet e34c87fda1ce4e18a8e10cb33464309a
I20260811 05:13:53.566871 20238 tablet_service.cc:1514] Processing DeleteTablet for tablet e34c87fda1ce4e18a8e10cb33464309a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:53 UTC) from {username='oryx', principal='oryx/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:57300
I20260811 05:13:53.566929 19764 tablet_service.cc:1514] Processing DeleteTablet for tablet e34c87fda1ce4e18a8e10cb33464309a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:53 UTC) from {username='oryx', principal='oryx/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:48188
I20260811 05:13:53.567056 20009 tablet_service.cc:1514] Processing DeleteTablet for tablet e34c87fda1ce4e18a8e10cb33464309a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:53 UTC) from {username='oryx', principal='oryx/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:45470
I20260811 05:13:53.567509 20439 tablet_replica.cc:330] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae: stopping tablet replica
I20260811 05:13:53.567641 20441 tablet_replica.cc:330] T e34c87fda1ce4e18a8e10cb33464309a P 2906ab24d87b4e97abae1f364a25e903: stopping tablet replica
I20260811 05:13:53.567938 20441 raft_consensus.cc:2229] T e34c87fda1ce4e18a8e10cb33464309a P 2906ab24d87b4e97abae1f364a25e903 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:53.567939 20439 raft_consensus.cc:2229] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:13:53.568320 19548 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 05:13:53.568653 20439 raft_consensus.cc:2258] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 05:13:53.568804 19548 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:53.569820 20441 raft_consensus.cc:2258] T e34c87fda1ce4e18a8e10cb33464309a P 2906ab24d87b4e97abae1f364a25e903 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:53.570353 20440 tablet_replica.cc:330] T e34c87fda1ce4e18a8e10cb33464309a P 945234845f614c6793619eee11183edc: stopping tablet replica
I20260811 05:13:53.570765 20440 raft_consensus.cc:2229] T e34c87fda1ce4e18a8e10cb33464309a P 945234845f614c6793619eee11183edc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:53.571158 20440 raft_consensus.cc:2258] T e34c87fda1ce4e18a8e10cb33464309a P 945234845f614c6793619eee11183edc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:53.571436 20441 ts_tablet_manager.cc:1905] T e34c87fda1ce4e18a8e10cb33464309a P 2906ab24d87b4e97abae1f364a25e903: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:53.572804 20440 ts_tablet_manager.cc:1905] T e34c87fda1ce4e18a8e10cb33464309a P 945234845f614c6793619eee11183edc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:53.574852 20441 ts_tablet_manager.cc:1918] T e34c87fda1ce4e18a8e10cb33464309a P 2906ab24d87b4e97abae1f364a25e903: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:53.575014 20441 log.cc:1196] T e34c87fda1ce4e18a8e10cb33464309a P 2906ab24d87b4e97abae1f364a25e903: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/e34c87fda1ce4e18a8e10cb33464309a
I20260811 05:13:53.575430 20441 ts_tablet_manager.cc:1939] T e34c87fda1ce4e18a8e10cb33464309a P 2906ab24d87b4e97abae1f364a25e903: Deleting consensus metadata
I20260811 05:13:53.575677 20238 tablet_service.cc:1467] Processing CreateTablet for tablet bd06f5d1be1944529766018418da7477 (DEFAULT_TABLE table=test-table [id=434c47be72f046d7a00f4cf7358f0c0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:53.575909 20440 ts_tablet_manager.cc:1918] T e34c87fda1ce4e18a8e10cb33464309a P 945234845f614c6793619eee11183edc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:53.576004 20440 log.cc:1196] T e34c87fda1ce4e18a8e10cb33464309a P 945234845f614c6793619eee11183edc: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-0/wal/wals/e34c87fda1ce4e18a8e10cb33464309a
I20260811 05:13:53.576026 20238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd06f5d1be1944529766018418da7477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:53.576368 20440 ts_tablet_manager.cc:1939] T e34c87fda1ce4e18a8e10cb33464309a P 945234845f614c6793619eee11183edc: Deleting consensus metadata
I20260811 05:13:53.578454 20404 tablet_bootstrap.cc:492] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae: Bootstrap starting.
I20260811 05:13:53.578951 19764 tablet_service.cc:1467] Processing CreateTablet for tablet bd06f5d1be1944529766018418da7477 (DEFAULT_TABLE table=test-table [id=434c47be72f046d7a00f4cf7358f0c0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:53.579226 19764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd06f5d1be1944529766018418da7477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:53.579567 20404 tablet_bootstrap.cc:654] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:53.580374 19523 catalog_manager.cc:4841] TS 945234845f614c6793619eee11183edc (127.7.67.129:46641): tablet e34c87fda1ce4e18a8e10cb33464309a (table test-table [id=0ab7b41be7b64376ac1ad71444bd34e7]) successfully deleted
I20260811 05:13:53.581053 19524 catalog_manager.cc:4841] TS 2906ab24d87b4e97abae1f364a25e903 (127.7.67.130:42179): tablet e34c87fda1ce4e18a8e10cb33464309a (table test-table [id=0ab7b41be7b64376ac1ad71444bd34e7]) successfully deleted
I20260811 05:13:53.581708 20009 tablet_service.cc:1467] Processing CreateTablet for tablet bd06f5d1be1944529766018418da7477 (DEFAULT_TABLE table=test-table [id=434c47be72f046d7a00f4cf7358f0c0f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:53.581846 20406 tablet_bootstrap.cc:492] T bd06f5d1be1944529766018418da7477 P 945234845f614c6793619eee11183edc: Bootstrap starting.
I20260811 05:13:53.581995 20009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd06f5d1be1944529766018418da7477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:53.582912 20406 tablet_bootstrap.cc:654] T bd06f5d1be1944529766018418da7477 P 945234845f614c6793619eee11183edc: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:53.584435 20406 tablet_bootstrap.cc:492] T bd06f5d1be1944529766018418da7477 P 945234845f614c6793619eee11183edc: No bootstrap required, opened a new log
I20260811 05:13:53.584555 20406 ts_tablet_manager.cc:1397] T bd06f5d1be1944529766018418da7477 P 945234845f614c6793619eee11183edc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260811 05:13:53.584637 20403 tablet_bootstrap.cc:492] T bd06f5d1be1944529766018418da7477 P 2906ab24d87b4e97abae1f364a25e903: Bootstrap starting.
I20260811 05:13:53.585390 20404 tablet_bootstrap.cc:492] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae: No bootstrap required, opened a new log
I20260811 05:13:53.585472 20404 ts_tablet_manager.cc:1397] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260811 05:13:53.585520 20403 tablet_bootstrap.cc:654] T bd06f5d1be1944529766018418da7477 P 2906ab24d87b4e97abae1f364a25e903: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:53.585512 20406 raft_consensus.cc:348] T bd06f5d1be1944529766018418da7477 P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.585634 20406 raft_consensus.cc:374] T bd06f5d1be1944529766018418da7477 P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:53.585668 20406 raft_consensus.cc:729] T bd06f5d1be1944529766018418da7477 P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 945234845f614c6793619eee11183edc, State: Initialized, Role: FOLLOWER
I20260811 05:13:53.585819 20406 consensus_queue.cc:260] T bd06f5d1be1944529766018418da7477 P 945234845f614c6793619eee11183edc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.585990 20404 raft_consensus.cc:348] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.586110 20404 raft_consensus.cc:374] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:53.586120 20406 ts_tablet_manager.cc:1428] T bd06f5d1be1944529766018418da7477 P 945234845f614c6793619eee11183edc: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:53.586169 20404 raft_consensus.cc:729] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73460f11a7024fc9843372b07ee57dae, State: Initialized, Role: FOLLOWER
I20260811 05:13:53.586355 20404 consensus_queue.cc:260] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.586658 20404 ts_tablet_manager.cc:1428] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:53.587539 20439 ts_tablet_manager.cc:1905] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:53.590553 20439 ts_tablet_manager.cc:1918] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:53.590687 20439 log.cc:1196] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/e34c87fda1ce4e18a8e10cb33464309a
I20260811 05:13:53.591069 20439 ts_tablet_manager.cc:1939] T e34c87fda1ce4e18a8e10cb33464309a P 73460f11a7024fc9843372b07ee57dae: Deleting consensus metadata
I20260811 05:13:53.592152 19525 catalog_manager.cc:4841] TS 73460f11a7024fc9843372b07ee57dae (127.7.67.131:41747): tablet e34c87fda1ce4e18a8e10cb33464309a (table test-table [id=0ab7b41be7b64376ac1ad71444bd34e7]) successfully deleted
I20260811 05:13:53.592938 20403 tablet_bootstrap.cc:492] T bd06f5d1be1944529766018418da7477 P 2906ab24d87b4e97abae1f364a25e903: No bootstrap required, opened a new log
I20260811 05:13:53.593027 20403 ts_tablet_manager.cc:1397] T bd06f5d1be1944529766018418da7477 P 2906ab24d87b4e97abae1f364a25e903: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260811 05:13:53.593459 20403 raft_consensus.cc:348] T bd06f5d1be1944529766018418da7477 P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.593570 20403 raft_consensus.cc:374] T bd06f5d1be1944529766018418da7477 P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:53.593601 20403 raft_consensus.cc:729] T bd06f5d1be1944529766018418da7477 P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2906ab24d87b4e97abae1f364a25e903, State: Initialized, Role: FOLLOWER
I20260811 05:13:53.593725 20403 consensus_queue.cc:260] T bd06f5d1be1944529766018418da7477 P 2906ab24d87b4e97abae1f364a25e903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.593971 20403 ts_tablet_manager.cc:1428] T bd06f5d1be1944529766018418da7477 P 2906ab24d87b4e97abae1f364a25e903: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 05:13:53.644480 20432 raft_consensus.cc:482] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:53.644690 20432 raft_consensus.cc:504] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.645183 20432 leader_election.cc:288] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 945234845f614c6793619eee11183edc (127.7.67.129:46641), 2906ab24d87b4e97abae1f364a25e903 (127.7.67.130:42179)
I20260811 05:13:53.645614 19793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd06f5d1be1944529766018418da7477" candidate_uuid: "73460f11a7024fc9843372b07ee57dae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945234845f614c6793619eee11183edc" is_pre_election: true
I20260811 05:13:53.645610 20041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd06f5d1be1944529766018418da7477" candidate_uuid: "73460f11a7024fc9843372b07ee57dae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2906ab24d87b4e97abae1f364a25e903" is_pre_election: true
I20260811 05:13:53.645782 19793 raft_consensus.cc:2454] T bd06f5d1be1944529766018418da7477 P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73460f11a7024fc9843372b07ee57dae in term 0.
I20260811 05:13:53.645809 20041 raft_consensus.cc:2454] T bd06f5d1be1944529766018418da7477 P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73460f11a7024fc9843372b07ee57dae in term 0.
I20260811 05:13:53.646067 20154 leader_election.cc:302] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73460f11a7024fc9843372b07ee57dae, 945234845f614c6793619eee11183edc; no voters:
I20260811 05:13:53.646243 20432 raft_consensus.cc:2781] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:53.646306 20432 raft_consensus.cc:482] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:53.646356 20432 raft_consensus.cc:3037] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:53.647397 20432 raft_consensus.cc:504] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.647786 20432 leader_election.cc:288] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae [CANDIDATE]: Term 1 election: Requested vote from peers 945234845f614c6793619eee11183edc (127.7.67.129:46641), 2906ab24d87b4e97abae1f364a25e903 (127.7.67.130:42179)
I20260811 05:13:53.648116 19793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd06f5d1be1944529766018418da7477" candidate_uuid: "73460f11a7024fc9843372b07ee57dae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945234845f614c6793619eee11183edc"
I20260811 05:13:53.648126 20041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd06f5d1be1944529766018418da7477" candidate_uuid: "73460f11a7024fc9843372b07ee57dae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2906ab24d87b4e97abae1f364a25e903"
I20260811 05:13:53.648238 19793 raft_consensus.cc:3037] T bd06f5d1be1944529766018418da7477 P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:53.648259 20041 raft_consensus.cc:3037] T bd06f5d1be1944529766018418da7477 P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:53.649219 20041 raft_consensus.cc:2454] T bd06f5d1be1944529766018418da7477 P 2906ab24d87b4e97abae1f364a25e903 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73460f11a7024fc9843372b07ee57dae in term 1.
I20260811 05:13:53.649453 19793 raft_consensus.cc:2454] T bd06f5d1be1944529766018418da7477 P 945234845f614c6793619eee11183edc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73460f11a7024fc9843372b07ee57dae in term 1.
I20260811 05:13:53.649516 20157 leader_election.cc:302] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2906ab24d87b4e97abae1f364a25e903, 73460f11a7024fc9843372b07ee57dae; no voters:
I20260811 05:13:53.649713 20432 raft_consensus.cc:2781] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:53.649812 20432 raft_consensus.cc:686] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae [term 1 LEADER]: Becoming Leader. State: Replica: 73460f11a7024fc9843372b07ee57dae, State: Running, Role: LEADER
I20260811 05:13:53.649950 20432 consensus_queue.cc:237] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.651633 19546 catalog_manager.cc:5495] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae reported cstate change: term changed from 0 to 1, leader changed from <none> to 73460f11a7024fc9843372b07ee57dae (127.7.67.131). New cstate: current_term: 1 leader_uuid: "73460f11a7024fc9843372b07ee57dae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } health_report { overall_health: HEALTHY } } }
Aug 11 05:13:53 dist-test-slave-04bb krb5kdc[19477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425232, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.190@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM
I20260811 05:13:53.688511 19623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.678428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52148 (local address 127.7.67.190:40745)
0811 05:13:53.678583 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:53.678587 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:53.678803 (+ 216us) server_negotiation.cc:407] Connection header received
0811 05:13:53.678925 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:53.678928 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:53.678971 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:53.679056 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:53.679988 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.680583 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.681772 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.682006 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.682100 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.684982 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:53.684985 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:53.684987 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:53.685010 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:53.686488 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:53.687054 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:53.687056 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:53.687058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:53.687131 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:53.687456 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:53.687459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:53.687460 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:53.687638 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:53.687715 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:53.688146 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 05:13:53.688362 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":106,"server-negotiator.queue_time_us":71}
I20260811 05:13:53.688546 20463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.678330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:40745 (local address 127.0.0.1:52148)
0811 05:13:53.678671 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:53.678690 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:53.678823 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:53.679118 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:53.679122 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:53.679705 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:53.679827 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.679837 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.680712 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.680715 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.681603 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.681616 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.682146 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.682152 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.682261 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.683005 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:53.683023 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:53.684829 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:53.686610 (+ 1781us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:53.686621 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:53.686628 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:53.686953 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:53.687243 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:53.687246 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:53.687248 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:53.687358 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:53.687764 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:53.687774 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:53.687964 (+ 190us) client_negotiation.cc:769] Sending connection context
0811 05:13:53.688099 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 05:13:53.688287 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":126,"threads_started":1}
I20260811 05:13:53.693084 19546 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:52148:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 05:13:53.694020 19546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:53.699689 19764 tablet_service.cc:1467] Processing CreateTablet for tablet 811eb3e80f77465bba0f9917a850bb27 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f407f61afb6b46508a3af340f12f0bdd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:13:53.699828 20009 tablet_service.cc:1467] Processing CreateTablet for tablet 811eb3e80f77465bba0f9917a850bb27 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f407f61afb6b46508a3af340f12f0bdd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:13:53.700129 19764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 811eb3e80f77465bba0f9917a850bb27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:53.700309 20009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 811eb3e80f77465bba0f9917a850bb27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:53.702744 20406 tablet_bootstrap.cc:492] T 811eb3e80f77465bba0f9917a850bb27 P 945234845f614c6793619eee11183edc: Bootstrap starting.
I20260811 05:13:53.703365 20238 tablet_service.cc:1467] Processing CreateTablet for tablet 811eb3e80f77465bba0f9917a850bb27 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f407f61afb6b46508a3af340f12f0bdd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 05:13:53.703787 20238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 811eb3e80f77465bba0f9917a850bb27. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:53.703821 20406 tablet_bootstrap.cc:654] T 811eb3e80f77465bba0f9917a850bb27 P 945234845f614c6793619eee11183edc: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:53.705090 20406 tablet_bootstrap.cc:492] T 811eb3e80f77465bba0f9917a850bb27 P 945234845f614c6793619eee11183edc: No bootstrap required, opened a new log
I20260811 05:13:53.705204 20406 ts_tablet_manager.cc:1397] T 811eb3e80f77465bba0f9917a850bb27 P 945234845f614c6793619eee11183edc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:13:53.705596 20406 raft_consensus.cc:348] T 811eb3e80f77465bba0f9917a850bb27 P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.705724 20406 raft_consensus.cc:374] T 811eb3e80f77465bba0f9917a850bb27 P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:53.705765 20406 raft_consensus.cc:729] T 811eb3e80f77465bba0f9917a850bb27 P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 945234845f614c6793619eee11183edc, State: Initialized, Role: FOLLOWER
I20260811 05:13:53.705889 20406 consensus_queue.cc:260] T 811eb3e80f77465bba0f9917a850bb27 P 945234845f614c6793619eee11183edc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.706179 20417 tablet_replica.cc:439] T 811eb3e80f77465bba0f9917a850bb27 P 945234845f614c6793619eee11183edc: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } } }
I20260811 05:13:53.706317 20417 tablet_replica.cc:442] T 811eb3e80f77465bba0f9917a850bb27 P 945234845f614c6793619eee11183edc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:53.706693 20404 tablet_bootstrap.cc:492] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae: Bootstrap starting.
I20260811 05:13:53.706753 20406 ts_tablet_manager.cc:1428] T 811eb3e80f77465bba0f9917a850bb27 P 945234845f614c6793619eee11183edc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:53.707527 20404 tablet_bootstrap.cc:654] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:53.708906 20403 tablet_bootstrap.cc:492] T 811eb3e80f77465bba0f9917a850bb27 P 2906ab24d87b4e97abae1f364a25e903: Bootstrap starting.
I20260811 05:13:53.709820 20403 tablet_bootstrap.cc:654] T 811eb3e80f77465bba0f9917a850bb27 P 2906ab24d87b4e97abae1f364a25e903: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:53.710793 20404 tablet_bootstrap.cc:492] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae: No bootstrap required, opened a new log
I20260811 05:13:53.710880 20404 ts_tablet_manager.cc:1397] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260811 05:13:53.711179 20403 tablet_bootstrap.cc:492] T 811eb3e80f77465bba0f9917a850bb27 P 2906ab24d87b4e97abae1f364a25e903: No bootstrap required, opened a new log
I20260811 05:13:53.711293 20403 ts_tablet_manager.cc:1397] T 811eb3e80f77465bba0f9917a850bb27 P 2906ab24d87b4e97abae1f364a25e903: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 05:13:53.711295 20404 raft_consensus.cc:348] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.711404 20404 raft_consensus.cc:374] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:53.711440 20404 raft_consensus.cc:729] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73460f11a7024fc9843372b07ee57dae, State: Initialized, Role: FOLLOWER
I20260811 05:13:53.711560 20404 consensus_queue.cc:260] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.711798 20404 ts_tablet_manager.cc:1428] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:53.711807 20403 raft_consensus.cc:348] T 811eb3e80f77465bba0f9917a850bb27 P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.711948 20403 raft_consensus.cc:374] T 811eb3e80f77465bba0f9917a850bb27 P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:53.711997 20403 raft_consensus.cc:729] T 811eb3e80f77465bba0f9917a850bb27 P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2906ab24d87b4e97abae1f364a25e903, State: Initialized, Role: FOLLOWER
I20260811 05:13:53.711967 20432 tablet_replica.cc:439] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } } }
I20260811 05:13:53.712085 20432 tablet_replica.cc:442] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:53.712568 20403 consensus_queue.cc:260] T 811eb3e80f77465bba0f9917a850bb27 P 2906ab24d87b4e97abae1f364a25e903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.712908 20403 ts_tablet_manager.cc:1428] T 811eb3e80f77465bba0f9917a850bb27 P 2906ab24d87b4e97abae1f364a25e903: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:53.712970 20414 tablet_replica.cc:439] T 811eb3e80f77465bba0f9917a850bb27 P 2906ab24d87b4e97abae1f364a25e903: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } } }
I20260811 05:13:53.713110 20414 tablet_replica.cc:442] T 811eb3e80f77465bba0f9917a850bb27 P 2906ab24d87b4e97abae1f364a25e903: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:53.715305 20432 raft_consensus.cc:482] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:53.715404 20432 raft_consensus.cc:504] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.715843 20432 leader_election.cc:288] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 945234845f614c6793619eee11183edc (127.7.67.129:46641), 2906ab24d87b4e97abae1f364a25e903 (127.7.67.130:42179)
I20260811 05:13:53.716127 19793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811eb3e80f77465bba0f9917a850bb27" candidate_uuid: "73460f11a7024fc9843372b07ee57dae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945234845f614c6793619eee11183edc" is_pre_election: true
I20260811 05:13:53.716257 19793 raft_consensus.cc:2454] T 811eb3e80f77465bba0f9917a850bb27 P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73460f11a7024fc9843372b07ee57dae in term 0.
I20260811 05:13:53.716344 20041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811eb3e80f77465bba0f9917a850bb27" candidate_uuid: "73460f11a7024fc9843372b07ee57dae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2906ab24d87b4e97abae1f364a25e903" is_pre_election: true
I20260811 05:13:53.716464 20041 raft_consensus.cc:2454] T 811eb3e80f77465bba0f9917a850bb27 P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73460f11a7024fc9843372b07ee57dae in term 0.
I20260811 05:13:53.716549 20154 leader_election.cc:302] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73460f11a7024fc9843372b07ee57dae, 945234845f614c6793619eee11183edc; no voters:
I20260811 05:13:53.716706 20432 raft_consensus.cc:2781] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:53.716781 20432 raft_consensus.cc:482] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:53.716830 20432 raft_consensus.cc:3037] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:53.717700 20432 raft_consensus.cc:504] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.718057 20432 leader_election.cc:288] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae [CANDIDATE]: Term 1 election: Requested vote from peers 945234845f614c6793619eee11183edc (127.7.67.129:46641), 2906ab24d87b4e97abae1f364a25e903 (127.7.67.130:42179)
I20260811 05:13:53.718377 19793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811eb3e80f77465bba0f9917a850bb27" candidate_uuid: "73460f11a7024fc9843372b07ee57dae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945234845f614c6793619eee11183edc"
I20260811 05:13:53.718384 20041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "811eb3e80f77465bba0f9917a850bb27" candidate_uuid: "73460f11a7024fc9843372b07ee57dae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2906ab24d87b4e97abae1f364a25e903"
I20260811 05:13:53.718492 20041 raft_consensus.cc:3037] T 811eb3e80f77465bba0f9917a850bb27 P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:53.718492 19793 raft_consensus.cc:3037] T 811eb3e80f77465bba0f9917a850bb27 P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:53.719542 20041 raft_consensus.cc:2454] T 811eb3e80f77465bba0f9917a850bb27 P 2906ab24d87b4e97abae1f364a25e903 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73460f11a7024fc9843372b07ee57dae in term 1.
I20260811 05:13:53.719564 19793 raft_consensus.cc:2454] T 811eb3e80f77465bba0f9917a850bb27 P 945234845f614c6793619eee11183edc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73460f11a7024fc9843372b07ee57dae in term 1.
I20260811 05:13:53.719818 20154 leader_election.cc:302] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73460f11a7024fc9843372b07ee57dae, 945234845f614c6793619eee11183edc; no voters:
I20260811 05:13:53.719964 20432 raft_consensus.cc:2781] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:53.720065 20432 raft_consensus.cc:686] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae [term 1 LEADER]: Becoming Leader. State: Replica: 73460f11a7024fc9843372b07ee57dae, State: Running, Role: LEADER
I20260811 05:13:53.720208 20432 consensus_queue.cc:237] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.721031 20432 tablet_replica.cc:439] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae: TxnStatusTablet state changed. Reason: New leader 73460f11a7024fc9843372b07ee57dae. Latest consensus state: current_term: 1 leader_uuid: "73460f11a7024fc9843372b07ee57dae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } } }
I20260811 05:13:53.721127 20432 tablet_replica.cc:442] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:13:53.721424 20472 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 05:13:53.721560 20472 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 05:13:53.722076 19548 catalog_manager.cc:5495] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae reported cstate change: term changed from 0 to 1, leader changed from <none> to 73460f11a7024fc9843372b07ee57dae (127.7.67.131). New cstate: current_term: 1 leader_uuid: "73460f11a7024fc9843372b07ee57dae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } health_report { overall_health: HEALTHY } } }
I20260811 05:13:53.734002 20463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.730589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:41747 (local address 127.0.0.1:57318)
0811 05:13:53.730741 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:53.730761 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:53.730862 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:53.731144 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:53.731146 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:53.731154 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:13:53.731269 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.731274 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.732149 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.732151 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.732831 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.732841 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.733348 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.733351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.733416 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.733462 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:13:53.733764 (+ 302us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:13:53.733776 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 05:13:53.733826 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 05:13:53.733894 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 05:13:53.734045 20396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.730699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57318 (local address 127.7.67.131:41747)
0811 05:13:53.730834 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:53.730838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:53.730853 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:13:53.730981 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:53.730984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:53.731035 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:53.731119 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:53.731374 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.732039 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.732945 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.733232 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.733332 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.733593 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:53.733673 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:53.733748 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:53.733865 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 05:13:53.733919 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 05:13:53.736410 20041 raft_consensus.cc:1261] T 811eb3e80f77465bba0f9917a850bb27 P 2906ab24d87b4e97abae1f364a25e903 [term 1 FOLLOWER]: Refusing update from remote peer 73460f11a7024fc9843372b07ee57dae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:53.736428 19793 raft_consensus.cc:1261] T 811eb3e80f77465bba0f9917a850bb27 P 945234845f614c6793619eee11183edc [term 1 FOLLOWER]: Refusing update from remote peer 73460f11a7024fc9843372b07ee57dae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:53.736936 20432 consensus_queue.cc:1035] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae [LEADER]: Connected to new peer: Peer: permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:53.737125 20416 consensus_queue.cc:1035] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae [LEADER]: Connected to new peer: Peer: permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:53.738516 20417 tablet_replica.cc:439] T 811eb3e80f77465bba0f9917a850bb27 P 945234845f614c6793619eee11183edc: TxnStatusTablet state changed. Reason: New leader 73460f11a7024fc9843372b07ee57dae. Latest consensus state: current_term: 1 leader_uuid: "73460f11a7024fc9843372b07ee57dae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } } }
I20260811 05:13:53.738626 20417 tablet_replica.cc:442] T 811eb3e80f77465bba0f9917a850bb27 P 945234845f614c6793619eee11183edc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:53.738873 20414 tablet_replica.cc:439] T 811eb3e80f77465bba0f9917a850bb27 P 2906ab24d87b4e97abae1f364a25e903: TxnStatusTablet state changed. Reason: New leader 73460f11a7024fc9843372b07ee57dae. Latest consensus state: current_term: 1 leader_uuid: "73460f11a7024fc9843372b07ee57dae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } } }
I20260811 05:13:53.738981 20414 tablet_replica.cc:442] T 811eb3e80f77465bba0f9917a850bb27 P 2906ab24d87b4e97abae1f364a25e903: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:53.740023 20432 tablet_replica.cc:439] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "73460f11a7024fc9843372b07ee57dae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } } }
I20260811 05:13:53.740114 20432 tablet_replica.cc:442] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:13:53.740206 20424 mvcc.cc:205] Tried to move back new op lower bound from 7317197757380354048 to 7317197757319614464. Current Snapshot: MvccSnapshot[applied={T|T < 7317197757380354048}]
I20260811 05:13:53.741555 20427 mvcc.cc:205] Tried to move back new op lower bound from 7317197757380354048 to 7317197757319614464. Current Snapshot: MvccSnapshot[applied={T|T < 7317197757380354048}]
I20260811 05:13:53.741699 20414 tablet_replica.cc:439] T 811eb3e80f77465bba0f9917a850bb27 P 2906ab24d87b4e97abae1f364a25e903: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "73460f11a7024fc9843372b07ee57dae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } } }
I20260811 05:13:53.741793 20414 tablet_replica.cc:442] T 811eb3e80f77465bba0f9917a850bb27 P 2906ab24d87b4e97abae1f364a25e903: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:53.744812 20420 tablet_replica.cc:439] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "73460f11a7024fc9843372b07ee57dae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } } }
I20260811 05:13:53.744935 20420 tablet_replica.cc:442] T 811eb3e80f77465bba0f9917a850bb27 P 73460f11a7024fc9843372b07ee57dae: This TxnStatusTablet replica's current role is: LEADER
I20260811 05:13:53.745884 20417 tablet_replica.cc:439] T 811eb3e80f77465bba0f9917a850bb27 P 945234845f614c6793619eee11183edc: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "73460f11a7024fc9843372b07ee57dae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } } }
I20260811 05:13:53.745987 20417 tablet_replica.cc:442] T 811eb3e80f77465bba0f9917a850bb27 P 945234845f614c6793619eee11183edc: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 05:13:53.755504 20362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.751983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:41747 (local address 127.7.67.131:40545)
0811 05:13:53.752132 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:53.752147 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:53.752237 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:53.752601 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:53.752604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:53.752612 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:13:53.752700 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.752705 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.753692 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.753695 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.754283 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.754290 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.754825 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.754827 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.754918 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.754961 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:13:53.755250 (+ 289us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:13:53.755256 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 05:13:53.755307 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 05:13:53.755371 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":2}
I20260811 05:13:53.755534 20396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.752086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:40545 (local address 127.7.67.131:41747)
0811 05:13:53.752361 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:53.752365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:53.752374 (+ 9us) server_negotiation.cc:407] Connection header received
0811 05:13:53.752406 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:53.752408 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:53.752444 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:53.752506 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:53.752810 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.753566 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.754392 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.754715 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.754814 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.755078 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:53.755153 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:53.755234 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:53.755341 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 05:13:53.755390 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":221}
I20260811 05:13:53.761874 19793 raft_consensus.cc:1261] T bd06f5d1be1944529766018418da7477 P 945234845f614c6793619eee11183edc [term 1 FOLLOWER]: Refusing update from remote peer 73460f11a7024fc9843372b07ee57dae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:53.761958 20041 raft_consensus.cc:1261] T bd06f5d1be1944529766018418da7477 P 2906ab24d87b4e97abae1f364a25e903 [term 1 FOLLOWER]: Refusing update from remote peer 73460f11a7024fc9843372b07ee57dae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:53.762369 20416 consensus_queue.cc:1035] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae [LEADER]: Connected to new peer: Peer: permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:53.762574 20420 consensus_queue.cc:1035] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae [LEADER]: Connected to new peer: Peer: permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:53.825716 19546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:53.825899 19546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:53.828224 19546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09: Sending DeleteTablet for 3 replicas of tablet bd06f5d1be1944529766018418da7477
I20260811 05:13:53.828924 20238 tablet_service.cc:1514] Processing DeleteTablet for tablet bd06f5d1be1944529766018418da7477 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:53 UTC) from {username='oryx', principal='oryx/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:57300
I20260811 05:13:53.828931 19764 tablet_service.cc:1514] Processing DeleteTablet for tablet bd06f5d1be1944529766018418da7477 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:53 UTC) from {username='oryx', principal='oryx/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:48188
I20260811 05:13:53.829124 20439 tablet_replica.cc:330] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae: stopping tablet replica
I20260811 05:13:53.829136 20009 tablet_service.cc:1514] Processing DeleteTablet for tablet bd06f5d1be1944529766018418da7477 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:53 UTC) from {username='oryx', principal='oryx/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:45470
I20260811 05:13:53.829267 20439 raft_consensus.cc:2229] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:13:53.829329 20440 tablet_replica.cc:330] T bd06f5d1be1944529766018418da7477 P 945234845f614c6793619eee11183edc: stopping tablet replica
I20260811 05:13:53.829435 20440 raft_consensus.cc:2229] T bd06f5d1be1944529766018418da7477 P 945234845f614c6793619eee11183edc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:53.829505 20439 raft_consensus.cc:2258] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:53.829551 20440 raft_consensus.cc:2258] T bd06f5d1be1944529766018418da7477 P 945234845f614c6793619eee11183edc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:53.829711 20441 tablet_replica.cc:330] T bd06f5d1be1944529766018418da7477 P 2906ab24d87b4e97abae1f364a25e903: stopping tablet replica
I20260811 05:13:53.829816 20441 raft_consensus.cc:2229] T bd06f5d1be1944529766018418da7477 P 2906ab24d87b4e97abae1f364a25e903 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:53.829929 20441 raft_consensus.cc:2258] T bd06f5d1be1944529766018418da7477 P 2906ab24d87b4e97abae1f364a25e903 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:53.830720 20441 ts_tablet_manager.cc:1905] T bd06f5d1be1944529766018418da7477 P 2906ab24d87b4e97abae1f364a25e903: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:53.830801 20440 ts_tablet_manager.cc:1905] T bd06f5d1be1944529766018418da7477 P 945234845f614c6793619eee11183edc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:53.833456 20439 ts_tablet_manager.cc:1905] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:53.833657 20440 ts_tablet_manager.cc:1918] T bd06f5d1be1944529766018418da7477 P 945234845f614c6793619eee11183edc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:13:53.833722 20440 log.cc:1196] T bd06f5d1be1944529766018418da7477 P 945234845f614c6793619eee11183edc: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-0/wal/wals/bd06f5d1be1944529766018418da7477
I20260811 05:13:53.833973 20440 ts_tablet_manager.cc:1939] T bd06f5d1be1944529766018418da7477 P 945234845f614c6793619eee11183edc: Deleting consensus metadata
I20260811 05:13:53.834662 19523 catalog_manager.cc:4841] TS 945234845f614c6793619eee11183edc (127.7.67.129:46641): tablet bd06f5d1be1944529766018418da7477 (table test-table [id=434c47be72f046d7a00f4cf7358f0c0f]) successfully deleted
I20260811 05:13:53.836588 20439 ts_tablet_manager.cc:1918] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:13:53.836692 20439 log.cc:1196] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/bd06f5d1be1944529766018418da7477
I20260811 05:13:53.836934 20439 ts_tablet_manager.cc:1939] T bd06f5d1be1944529766018418da7477 P 73460f11a7024fc9843372b07ee57dae: Deleting consensus metadata
I20260811 05:13:53.837599 19525 catalog_manager.cc:4841] TS 73460f11a7024fc9843372b07ee57dae (127.7.67.131:41747): tablet bd06f5d1be1944529766018418da7477 (table test-table [id=434c47be72f046d7a00f4cf7358f0c0f]) successfully deleted
I20260811 05:13:53.839076 20441 ts_tablet_manager.cc:1918] T bd06f5d1be1944529766018418da7477 P 2906ab24d87b4e97abae1f364a25e903: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:13:53.839186 20441 log.cc:1196] T bd06f5d1be1944529766018418da7477 P 2906ab24d87b4e97abae1f364a25e903: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/bd06f5d1be1944529766018418da7477
I20260811 05:13:53.839484 20441 ts_tablet_manager.cc:1939] T bd06f5d1be1944529766018418da7477 P 2906ab24d87b4e97abae1f364a25e903: Deleting consensus metadata
I20260811 05:13:53.840256 19524 catalog_manager.cc:4841] TS 2906ab24d87b4e97abae1f364a25e903 (127.7.67.130:42179): tablet bd06f5d1be1944529766018418da7477 (table test-table [id=434c47be72f046d7a00f4cf7358f0c0f]) successfully deleted
Aug 11 05:13:53 dist-test-slave-04bb krb5kdc[19477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 05:13:53 dist-test-slave-04bb krb5kdc[19477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:53 dist-test-slave-04bb krb5kdc[19477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 05:13:53.860415 19623 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:53.852674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52154 (local address 127.7.67.190:40745)
0811 05:13:53.852831 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:53.852836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:53.853034 (+ 198us) server_negotiation.cc:407] Connection header received
0811 05:13:53.853228 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:53.853231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:53.853279 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:53.853345 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:53.854211 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.854743 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.856009 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.856228 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.856316 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.860242 (+ 3926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52154: BlockingRecv error: recv got EOF from 127.0.0.1:52154 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 05:13:53 dist-test-slave-04bb krb5kdc[19477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425233, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM
I20260811 05:13:53.871320 19623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.862081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52168 (local address 127.7.67.190:40745)
0811 05:13:53.862222 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:53.862227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:53.862374 (+ 147us) server_negotiation.cc:407] Connection header received
0811 05:13:53.862531 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:53.862534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:53.862578 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:53.862646 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:53.863451 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.864010 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.865180 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.865406 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.865494 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.868031 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:53.868034 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:53.868036 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:53.868059 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:53.869557 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:53.870052 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:53.870056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:53.870057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:53.870142 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:53.870449 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:53.870453 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:53.870454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:53.870642 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:53.870794 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:53.871036 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 05:13:53.871148 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 05:13:53.873988 19546 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52168:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:13:53.874464 19546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:53.879547 20009 tablet_service.cc:1467] Processing CreateTablet for tablet fcefa7e8e82c41e3939334d42c609dd9 (DEFAULT_TABLE table=test-table [id=af91e360f849464696bc83218cee06fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:53.879638 20238 tablet_service.cc:1467] Processing CreateTablet for tablet fcefa7e8e82c41e3939334d42c609dd9 (DEFAULT_TABLE table=test-table [id=af91e360f849464696bc83218cee06fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:53.879681 19764 tablet_service.cc:1467] Processing CreateTablet for tablet fcefa7e8e82c41e3939334d42c609dd9 (DEFAULT_TABLE table=test-table [id=af91e360f849464696bc83218cee06fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:53.879899 20238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcefa7e8e82c41e3939334d42c609dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:53.879899 20009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcefa7e8e82c41e3939334d42c609dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:53.879964 19764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcefa7e8e82c41e3939334d42c609dd9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:53.882488 20403 tablet_bootstrap.cc:492] T fcefa7e8e82c41e3939334d42c609dd9 P 2906ab24d87b4e97abae1f364a25e903: Bootstrap starting.
I20260811 05:13:53.882504 20404 tablet_bootstrap.cc:492] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae: Bootstrap starting.
I20260811 05:13:53.883339 20403 tablet_bootstrap.cc:654] T fcefa7e8e82c41e3939334d42c609dd9 P 2906ab24d87b4e97abae1f364a25e903: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:53.883343 20404 tablet_bootstrap.cc:654] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:53.883896 20406 tablet_bootstrap.cc:492] T fcefa7e8e82c41e3939334d42c609dd9 P 945234845f614c6793619eee11183edc: Bootstrap starting.
I20260811 05:13:53.884788 20406 tablet_bootstrap.cc:654] T fcefa7e8e82c41e3939334d42c609dd9 P 945234845f614c6793619eee11183edc: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:53.885573 20404 tablet_bootstrap.cc:492] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae: No bootstrap required, opened a new log
I20260811 05:13:53.885671 20404 ts_tablet_manager.cc:1397] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 05:13:53.886073 20406 tablet_bootstrap.cc:492] T fcefa7e8e82c41e3939334d42c609dd9 P 945234845f614c6793619eee11183edc: No bootstrap required, opened a new log
I20260811 05:13:53.886164 20406 ts_tablet_manager.cc:1397] T fcefa7e8e82c41e3939334d42c609dd9 P 945234845f614c6793619eee11183edc: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:53.886147 20404 raft_consensus.cc:348] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.886261 20404 raft_consensus.cc:374] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:53.886291 20404 raft_consensus.cc:729] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73460f11a7024fc9843372b07ee57dae, State: Initialized, Role: FOLLOWER
I20260811 05:13:53.886430 20404 consensus_queue.cc:260] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.886616 20406 raft_consensus.cc:348] T fcefa7e8e82c41e3939334d42c609dd9 P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.886736 20406 raft_consensus.cc:374] T fcefa7e8e82c41e3939334d42c609dd9 P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:53.886710 20404 ts_tablet_manager.cc:1428] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:53.886767 20406 raft_consensus.cc:729] T fcefa7e8e82c41e3939334d42c609dd9 P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 945234845f614c6793619eee11183edc, State: Initialized, Role: FOLLOWER
I20260811 05:13:53.886889 20406 consensus_queue.cc:260] T fcefa7e8e82c41e3939334d42c609dd9 P 945234845f614c6793619eee11183edc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.887125 20406 ts_tablet_manager.cc:1428] T fcefa7e8e82c41e3939334d42c609dd9 P 945234845f614c6793619eee11183edc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:53.888978 20403 tablet_bootstrap.cc:492] T fcefa7e8e82c41e3939334d42c609dd9 P 2906ab24d87b4e97abae1f364a25e903: No bootstrap required, opened a new log
I20260811 05:13:53.889078 20403 ts_tablet_manager.cc:1397] T fcefa7e8e82c41e3939334d42c609dd9 P 2906ab24d87b4e97abae1f364a25e903: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260811 05:13:53.889505 20403 raft_consensus.cc:348] T fcefa7e8e82c41e3939334d42c609dd9 P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.889632 20403 raft_consensus.cc:374] T fcefa7e8e82c41e3939334d42c609dd9 P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:53.889690 20403 raft_consensus.cc:729] T fcefa7e8e82c41e3939334d42c609dd9 P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2906ab24d87b4e97abae1f364a25e903, State: Initialized, Role: FOLLOWER
I20260811 05:13:53.889822 20403 consensus_queue.cc:260] T fcefa7e8e82c41e3939334d42c609dd9 P 2906ab24d87b4e97abae1f364a25e903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.889878 20420 raft_consensus.cc:482] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:53.889950 20420 raft_consensus.cc:504] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.890108 20403 ts_tablet_manager.cc:1428] T fcefa7e8e82c41e3939334d42c609dd9 P 2906ab24d87b4e97abae1f364a25e903: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:53.890411 20420 leader_election.cc:288] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 945234845f614c6793619eee11183edc (127.7.67.129:46641), 2906ab24d87b4e97abae1f364a25e903 (127.7.67.130:42179)
I20260811 05:13:53.890929 20041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcefa7e8e82c41e3939334d42c609dd9" candidate_uuid: "73460f11a7024fc9843372b07ee57dae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2906ab24d87b4e97abae1f364a25e903" is_pre_election: true
I20260811 05:13:53.891053 20041 raft_consensus.cc:2454] T fcefa7e8e82c41e3939334d42c609dd9 P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73460f11a7024fc9843372b07ee57dae in term 0.
I20260811 05:13:53.891271 19793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcefa7e8e82c41e3939334d42c609dd9" candidate_uuid: "73460f11a7024fc9843372b07ee57dae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945234845f614c6793619eee11183edc" is_pre_election: true
I20260811 05:13:53.891390 19793 raft_consensus.cc:2454] T fcefa7e8e82c41e3939334d42c609dd9 P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73460f11a7024fc9843372b07ee57dae in term 0.
I20260811 05:13:53.891372 20157 leader_election.cc:302] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2906ab24d87b4e97abae1f364a25e903, 73460f11a7024fc9843372b07ee57dae; no voters:
I20260811 05:13:53.891534 20420 raft_consensus.cc:2781] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:53.891590 20420 raft_consensus.cc:482] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:53.891628 20420 raft_consensus.cc:3037] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:53.892513 20420 raft_consensus.cc:504] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.892911 20420 leader_election.cc:288] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae [CANDIDATE]: Term 1 election: Requested vote from peers 945234845f614c6793619eee11183edc (127.7.67.129:46641), 2906ab24d87b4e97abae1f364a25e903 (127.7.67.130:42179)
I20260811 05:13:53.893242 20041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcefa7e8e82c41e3939334d42c609dd9" candidate_uuid: "73460f11a7024fc9843372b07ee57dae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2906ab24d87b4e97abae1f364a25e903"
I20260811 05:13:53.893325 19793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcefa7e8e82c41e3939334d42c609dd9" candidate_uuid: "73460f11a7024fc9843372b07ee57dae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945234845f614c6793619eee11183edc"
I20260811 05:13:53.893365 20041 raft_consensus.cc:3037] T fcefa7e8e82c41e3939334d42c609dd9 P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:53.893441 19793 raft_consensus.cc:3037] T fcefa7e8e82c41e3939334d42c609dd9 P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:53.894502 20041 raft_consensus.cc:2454] T fcefa7e8e82c41e3939334d42c609dd9 P 2906ab24d87b4e97abae1f364a25e903 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73460f11a7024fc9843372b07ee57dae in term 1.
I20260811 05:13:53.894783 20157 leader_election.cc:302] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2906ab24d87b4e97abae1f364a25e903, 73460f11a7024fc9843372b07ee57dae; no voters:
I20260811 05:13:53.894972 20420 raft_consensus.cc:2781] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:53.895077 20420 raft_consensus.cc:686] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae [term 1 LEADER]: Becoming Leader. State: Replica: 73460f11a7024fc9843372b07ee57dae, State: Running, Role: LEADER
I20260811 05:13:53.895156 19793 raft_consensus.cc:2454] T fcefa7e8e82c41e3939334d42c609dd9 P 945234845f614c6793619eee11183edc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73460f11a7024fc9843372b07ee57dae in term 1.
I20260811 05:13:53.895200 20420 consensus_queue.cc:237] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.896868 19546 catalog_manager.cc:5495] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae reported cstate change: term changed from 0 to 1, leader changed from <none> to 73460f11a7024fc9843372b07ee57dae (127.7.67.131). New cstate: current_term: 1 leader_uuid: "73460f11a7024fc9843372b07ee57dae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } health_report { overall_health: HEALTHY } } }
I20260811 05:13:53.906911 20396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.903599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57332 (local address 127.7.67.131:41747)
0811 05:13:53.903728 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:53.903733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:53.903748 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:13:53.903849 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:53.903852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:53.903904 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:53.903991 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:53.904255 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.904833 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.905995 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.906198 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.906277 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.906494 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:53.906558 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:53.906609 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:53.906709 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 05:13:53.906756 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 05:13:53.908738 20041 raft_consensus.cc:1261] T fcefa7e8e82c41e3939334d42c609dd9 P 2906ab24d87b4e97abae1f364a25e903 [term 1 FOLLOWER]: Refusing update from remote peer 73460f11a7024fc9843372b07ee57dae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:53.908757 19793 raft_consensus.cc:1261] T fcefa7e8e82c41e3939334d42c609dd9 P 945234845f614c6793619eee11183edc [term 1 FOLLOWER]: Refusing update from remote peer 73460f11a7024fc9843372b07ee57dae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:53.909145 20420 consensus_queue.cc:1035] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae [LEADER]: Connected to new peer: Peer: permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:53.909332 20416 consensus_queue.cc:1035] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae [LEADER]: Connected to new peer: Peer: permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:53.916006 19546 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:53.916150 19546 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:53.918278 19546 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09: Sending DeleteTablet for 3 replicas of tablet fcefa7e8e82c41e3939334d42c609dd9
I20260811 05:13:53.918869 20238 tablet_service.cc:1514] Processing DeleteTablet for tablet fcefa7e8e82c41e3939334d42c609dd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:53 UTC) from {username='oryx', principal='oryx/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:57300
I20260811 05:13:53.918895 20009 tablet_service.cc:1514] Processing DeleteTablet for tablet fcefa7e8e82c41e3939334d42c609dd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:53 UTC) from {username='oryx', principal='oryx/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:45470
I20260811 05:13:53.919047 20439 tablet_replica.cc:330] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae: stopping tablet replica
I20260811 05:13:53.919063 19764 tablet_service.cc:1514] Processing DeleteTablet for tablet fcefa7e8e82c41e3939334d42c609dd9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:53 UTC) from {username='oryx', principal='oryx/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:48188
I20260811 05:13:53.919133 20439 raft_consensus.cc:2229] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:13:53.919296 20439 raft_consensus.cc:2258] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:53.919296 20440 tablet_replica.cc:330] T fcefa7e8e82c41e3939334d42c609dd9 P 945234845f614c6793619eee11183edc: stopping tablet replica
I20260811 05:13:53.919432 20440 raft_consensus.cc:2229] T fcefa7e8e82c41e3939334d42c609dd9 P 945234845f614c6793619eee11183edc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:53.919474 20441 tablet_replica.cc:330] T fcefa7e8e82c41e3939334d42c609dd9 P 2906ab24d87b4e97abae1f364a25e903: stopping tablet replica
I20260811 05:13:53.919559 20440 raft_consensus.cc:2258] T fcefa7e8e82c41e3939334d42c609dd9 P 945234845f614c6793619eee11183edc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:53.919586 20441 raft_consensus.cc:2229] T fcefa7e8e82c41e3939334d42c609dd9 P 2906ab24d87b4e97abae1f364a25e903 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:53.919701 20441 raft_consensus.cc:2258] T fcefa7e8e82c41e3939334d42c609dd9 P 2906ab24d87b4e97abae1f364a25e903 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:53.919651 19546 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52168:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:13:53.920059 19546 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:13:53.920459 20441 ts_tablet_manager.cc:1905] T fcefa7e8e82c41e3939334d42c609dd9 P 2906ab24d87b4e97abae1f364a25e903: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:53.922838 20441 ts_tablet_manager.cc:1918] T fcefa7e8e82c41e3939334d42c609dd9 P 2906ab24d87b4e97abae1f364a25e903: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:53.922923 20441 log.cc:1196] T fcefa7e8e82c41e3939334d42c609dd9 P 2906ab24d87b4e97abae1f364a25e903: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/fcefa7e8e82c41e3939334d42c609dd9
I20260811 05:13:53.923259 20441 ts_tablet_manager.cc:1939] T fcefa7e8e82c41e3939334d42c609dd9 P 2906ab24d87b4e97abae1f364a25e903: Deleting consensus metadata
I20260811 05:13:53.923944 19524 catalog_manager.cc:4841] TS 2906ab24d87b4e97abae1f364a25e903 (127.7.67.130:42179): tablet fcefa7e8e82c41e3939334d42c609dd9 (table test-table [id=af91e360f849464696bc83218cee06fa]) successfully deleted
I20260811 05:13:53.924257 20439 ts_tablet_manager.cc:1905] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:53.924968 19764 tablet_service.cc:1467] Processing CreateTablet for tablet 4f8815d2091e4d79ada65ae72c064e86 (DEFAULT_TABLE table=test-table [id=7bed81586c6b45b099ea439db13c011e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:53.925307 19764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f8815d2091e4d79ada65ae72c064e86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:53.926342 20439 ts_tablet_manager.cc:1918] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:53.926442 20439 log.cc:1196] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/fcefa7e8e82c41e3939334d42c609dd9
I20260811 05:13:53.926671 20439 ts_tablet_manager.cc:1939] T fcefa7e8e82c41e3939334d42c609dd9 P 73460f11a7024fc9843372b07ee57dae: Deleting consensus metadata
I20260811 05:13:53.927253 19525 catalog_manager.cc:4841] TS 73460f11a7024fc9843372b07ee57dae (127.7.67.131:41747): tablet fcefa7e8e82c41e3939334d42c609dd9 (table test-table [id=af91e360f849464696bc83218cee06fa]) successfully deleted
I20260811 05:13:53.927881 20440 ts_tablet_manager.cc:1905] T fcefa7e8e82c41e3939334d42c609dd9 P 945234845f614c6793619eee11183edc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:53.928032 20406 tablet_bootstrap.cc:492] T 4f8815d2091e4d79ada65ae72c064e86 P 945234845f614c6793619eee11183edc: Bootstrap starting.
I20260811 05:13:53.928258 20238 tablet_service.cc:1467] Processing CreateTablet for tablet 4f8815d2091e4d79ada65ae72c064e86 (DEFAULT_TABLE table=test-table [id=7bed81586c6b45b099ea439db13c011e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:53.928546 20238 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f8815d2091e4d79ada65ae72c064e86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:53.929112 20406 tablet_bootstrap.cc:654] T 4f8815d2091e4d79ada65ae72c064e86 P 945234845f614c6793619eee11183edc: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:53.929857 20440 ts_tablet_manager.cc:1918] T fcefa7e8e82c41e3939334d42c609dd9 P 945234845f614c6793619eee11183edc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:13:53.929942 20440 log.cc:1196] T fcefa7e8e82c41e3939334d42c609dd9 P 945234845f614c6793619eee11183edc: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-0/wal/wals/fcefa7e8e82c41e3939334d42c609dd9
I20260811 05:13:53.930223 20440 ts_tablet_manager.cc:1939] T fcefa7e8e82c41e3939334d42c609dd9 P 945234845f614c6793619eee11183edc: Deleting consensus metadata
I20260811 05:13:53.930898 19523 catalog_manager.cc:4841] TS 945234845f614c6793619eee11183edc (127.7.67.129:46641): tablet fcefa7e8e82c41e3939334d42c609dd9 (table test-table [id=af91e360f849464696bc83218cee06fa]) successfully deleted
I20260811 05:13:53.930966 20009 tablet_service.cc:1467] Processing CreateTablet for tablet 4f8815d2091e4d79ada65ae72c064e86 (DEFAULT_TABLE table=test-table [id=7bed81586c6b45b099ea439db13c011e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:13:53.931227 20009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f8815d2091e4d79ada65ae72c064e86. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:53.932731 20404 tablet_bootstrap.cc:492] T 4f8815d2091e4d79ada65ae72c064e86 P 73460f11a7024fc9843372b07ee57dae: Bootstrap starting.
I20260811 05:13:53.933065 20403 tablet_bootstrap.cc:492] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903: Bootstrap starting.
I20260811 05:13:53.933597 20404 tablet_bootstrap.cc:654] T 4f8815d2091e4d79ada65ae72c064e86 P 73460f11a7024fc9843372b07ee57dae: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:53.934118 20403 tablet_bootstrap.cc:654] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:53.934744 20404 tablet_bootstrap.cc:492] T 4f8815d2091e4d79ada65ae72c064e86 P 73460f11a7024fc9843372b07ee57dae: No bootstrap required, opened a new log
I20260811 05:13:53.934790 20406 tablet_bootstrap.cc:492] T 4f8815d2091e4d79ada65ae72c064e86 P 945234845f614c6793619eee11183edc: No bootstrap required, opened a new log
I20260811 05:13:53.934845 20404 ts_tablet_manager.cc:1397] T 4f8815d2091e4d79ada65ae72c064e86 P 73460f11a7024fc9843372b07ee57dae: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 05:13:53.934870 20406 ts_tablet_manager.cc:1397] T 4f8815d2091e4d79ada65ae72c064e86 P 945234845f614c6793619eee11183edc: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.002s
I20260811 05:13:53.935331 20406 raft_consensus.cc:348] T 4f8815d2091e4d79ada65ae72c064e86 P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.935416 20406 raft_consensus.cc:374] T 4f8815d2091e4d79ada65ae72c064e86 P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:53.935437 20406 raft_consensus.cc:729] T 4f8815d2091e4d79ada65ae72c064e86 P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 945234845f614c6793619eee11183edc, State: Initialized, Role: FOLLOWER
I20260811 05:13:53.935371 20404 raft_consensus.cc:348] T 4f8815d2091e4d79ada65ae72c064e86 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.935487 20404 raft_consensus.cc:374] T 4f8815d2091e4d79ada65ae72c064e86 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:53.935525 20404 raft_consensus.cc:729] T 4f8815d2091e4d79ada65ae72c064e86 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73460f11a7024fc9843372b07ee57dae, State: Initialized, Role: FOLLOWER
I20260811 05:13:53.935546 20403 tablet_bootstrap.cc:492] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903: No bootstrap required, opened a new log
I20260811 05:13:53.935629 20403 ts_tablet_manager.cc:1397] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:13:53.935597 20406 consensus_queue.cc:260] T 4f8815d2091e4d79ada65ae72c064e86 P 945234845f614c6793619eee11183edc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.935663 20404 consensus_queue.cc:260] T 4f8815d2091e4d79ada65ae72c064e86 P 73460f11a7024fc9843372b07ee57dae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.935906 20406 ts_tablet_manager.cc:1428] T 4f8815d2091e4d79ada65ae72c064e86 P 945234845f614c6793619eee11183edc: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 05:13:53.935968 20404 ts_tablet_manager.cc:1428] T 4f8815d2091e4d79ada65ae72c064e86 P 73460f11a7024fc9843372b07ee57dae: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:53.936139 20403 raft_consensus.cc:348] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.936262 20403 raft_consensus.cc:374] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:53.936313 20403 raft_consensus.cc:729] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2906ab24d87b4e97abae1f364a25e903, State: Initialized, Role: FOLLOWER
I20260811 05:13:53.936448 20403 consensus_queue.cc:260] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.936787 20403 ts_tablet_manager.cc:1428] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:13:53.963837 20414 raft_consensus.cc:482] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:13:53.963989 20414 raft_consensus.cc:504] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.965205 20414 leader_election.cc:288] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 945234845f614c6793619eee11183edc (127.7.67.129:46641), 73460f11a7024fc9843372b07ee57dae (127.7.67.131:41747)
I20260811 05:13:53.969581 20399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.965523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:58395 (local address 127.7.67.129:46641)
0811 05:13:53.965670 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:53.965675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:53.965836 (+ 161us) server_negotiation.cc:407] Connection header received
0811 05:13:53.966031 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:53.966035 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:53.966099 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:53.966163 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:53.966509 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.967081 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.968446 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.968942 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.969046 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.969165 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:53.969339 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 05:13:53.969442 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 05:13:53.969691 20396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.965561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:44517 (local address 127.7.67.131:41747)
0811 05:13:53.965716 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:53.965720 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:53.965908 (+ 188us) server_negotiation.cc:407] Connection header received
0811 05:13:53.966075 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:53.966078 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:53.966121 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:53.966217 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:13:53.966557 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.967047 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.968556 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.969059 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.969179 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.969230 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:53.969490 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 05:13:53.969565 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 05:13:53.969859 20525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.965454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:41747 (local address 127.7.67.130:44517)
0811 05:13:53.965825 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:53.965838 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:53.965934 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:53.966257 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:53.966261 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:53.966301 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:53.966409 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.966414 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.967170 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.967175 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.968330 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.968340 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.969220 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.969224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.969352 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.969383 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 05:13:53.969477 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 05:13:53.969552 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":26,"thread_start_us":133,"threads_started":1}
I20260811 05:13:53.970167 20523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:53.965415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:46641 (local address 127.7.67.130:58395)
0811 05:13:53.965721 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:53.965736 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:53.965883 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:53.966199 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:53.966206 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:53.966223 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:13:53.966386 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.966392 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.967254 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.967257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.968290 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:53.968302 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:53.969068 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:53.969072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:53.969182 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:53.969208 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 05:13:53.969933 (+ 725us) client_negotiation.cc:240] Negotiation successful
0811 05:13:53.970028 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":1,"thread_start_us":107,"threads_started":1}
I20260811 05:13:53.970455 20264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f8815d2091e4d79ada65ae72c064e86" candidate_uuid: "2906ab24d87b4e97abae1f364a25e903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73460f11a7024fc9843372b07ee57dae" is_pre_election: true
I20260811 05:13:53.970697 19793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f8815d2091e4d79ada65ae72c064e86" candidate_uuid: "2906ab24d87b4e97abae1f364a25e903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945234845f614c6793619eee11183edc" is_pre_election: true
I20260811 05:13:53.970714 20264 raft_consensus.cc:2454] T 4f8815d2091e4d79ada65ae72c064e86 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2906ab24d87b4e97abae1f364a25e903 in term 0.
I20260811 05:13:53.970872 19793 raft_consensus.cc:2454] T 4f8815d2091e4d79ada65ae72c064e86 P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2906ab24d87b4e97abae1f364a25e903 in term 0.
I20260811 05:13:53.971212 19931 leader_election.cc:302] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2906ab24d87b4e97abae1f364a25e903, 73460f11a7024fc9843372b07ee57dae, 945234845f614c6793619eee11183edc; no voters:
I20260811 05:13:53.971518 20414 raft_consensus.cc:2781] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:13:53.971589 20414 raft_consensus.cc:482] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:13:53.971654 20414 raft_consensus.cc:3037] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:53.972796 20414 raft_consensus.cc:504] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.973229 20414 leader_election.cc:288] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903 [CANDIDATE]: Term 1 election: Requested vote from peers 945234845f614c6793619eee11183edc (127.7.67.129:46641), 73460f11a7024fc9843372b07ee57dae (127.7.67.131:41747)
I20260811 05:13:53.973546 19793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f8815d2091e4d79ada65ae72c064e86" candidate_uuid: "2906ab24d87b4e97abae1f364a25e903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "945234845f614c6793619eee11183edc"
I20260811 05:13:53.973603 20264 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f8815d2091e4d79ada65ae72c064e86" candidate_uuid: "2906ab24d87b4e97abae1f364a25e903" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "73460f11a7024fc9843372b07ee57dae"
I20260811 05:13:53.973687 19793 raft_consensus.cc:3037] T 4f8815d2091e4d79ada65ae72c064e86 P 945234845f614c6793619eee11183edc [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:53.973722 20264 raft_consensus.cc:3037] T 4f8815d2091e4d79ada65ae72c064e86 P 73460f11a7024fc9843372b07ee57dae [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:53.974565 19793 raft_consensus.cc:2454] T 4f8815d2091e4d79ada65ae72c064e86 P 945234845f614c6793619eee11183edc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2906ab24d87b4e97abae1f364a25e903 in term 1.
I20260811 05:13:53.974732 20264 raft_consensus.cc:2454] T 4f8815d2091e4d79ada65ae72c064e86 P 73460f11a7024fc9843372b07ee57dae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2906ab24d87b4e97abae1f364a25e903 in term 1.
I20260811 05:13:53.974881 19929 leader_election.cc:302] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2906ab24d87b4e97abae1f364a25e903, 945234845f614c6793619eee11183edc; no voters:
I20260811 05:13:53.975063 20414 raft_consensus.cc:2781] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:53.975306 20414 raft_consensus.cc:686] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903 [term 1 LEADER]: Becoming Leader. State: Replica: 2906ab24d87b4e97abae1f364a25e903, State: Running, Role: LEADER
I20260811 05:13:53.975572 20414 consensus_queue.cc:237] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } }
I20260811 05:13:53.977809 19546 catalog_manager.cc:5495] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2906ab24d87b4e97abae1f364a25e903 (127.7.67.130). New cstate: current_term: 1 leader_uuid: "2906ab24d87b4e97abae1f364a25e903" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2906ab24d87b4e97abae1f364a25e903" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 42179 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 } health_report { overall_health: UNKNOWN } } }
I20260811 05:13:54.006026 20400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:54.002068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45480 (local address 127.7.67.130:42179)
0811 05:13:54.002195 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:54.002199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:54.002212 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:13:54.002367 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:54.002370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:54.002442 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:54.002559 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:54.002792 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:54.003424 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:54.004381 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:54.004665 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:54.004785 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:54.005065 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:54.005648 (+ 583us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:54.005740 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:54.005824 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 05:13:54.005871 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 05:13:54.016325 20396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:54.013079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:59149 (local address 127.7.67.131:41747)
0811 05:13:54.013222 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:54.013227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:54.013396 (+ 169us) server_negotiation.cc:407] Connection header received
0811 05:13:54.013500 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:54.013503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:54.013544 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:54.013599 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:54.013869 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:54.014429 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:54.015358 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:54.015589 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:54.015674 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:54.015954 (+ 280us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:54.016018 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:54.016076 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:54.016172 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 05:13:54.016219 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 05:13:54.016386 20533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:54.013003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:41747 (local address 127.7.67.130:59149)
0811 05:13:54.013313 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:54.013327 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:54.013419 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:54.013625 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:54.013627 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:54.013634 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:13:54.013731 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:54.013735 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:54.014547 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:54.014550 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:54.015219 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:54.015228 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:54.015699 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:54.015702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:54.015784 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:54.015828 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:13:54.016102 (+ 274us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:13:54.016108 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 05:13:54.016166 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 05:13:54.016232 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":109,"threads_started":1}
I20260811 05:13:54.022369 19793 raft_consensus.cc:1261] T 4f8815d2091e4d79ada65ae72c064e86 P 945234845f614c6793619eee11183edc [term 1 FOLLOWER]: Refusing update from remote peer 2906ab24d87b4e97abae1f364a25e903: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:54.022583 20264 raft_consensus.cc:1261] T 4f8815d2091e4d79ada65ae72c064e86 P 73460f11a7024fc9843372b07ee57dae [term 1 FOLLOWER]: Refusing update from remote peer 2906ab24d87b4e97abae1f364a25e903: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:13:54.022871 20528 consensus_queue.cc:1035] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903 [LEADER]: Connected to new peer: Peer: permanent_uuid: "945234845f614c6793619eee11183edc" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:54.023108 20414 consensus_queue.cc:1035] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903 [LEADER]: Connected to new peer: Peer: permanent_uuid: "73460f11a7024fc9843372b07ee57dae" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 41747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:13:54.051945 20400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:54.047915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:41673 (local address 127.7.67.130:42179)
0811 05:13:54.048055 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:54.048061 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:54.048083 (+ 22us) server_negotiation.cc:407] Connection header received
0811 05:13:54.048152 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:54.048155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:54.048232 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:54.048357 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:13:54.048752 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:54.049593 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:54.050899 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:54.051132 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:54.051235 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:54.051520 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:13:54.051596 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:13:54.051676 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:54.051791 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 05:13:54.051839 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 05:13:54.052000 20362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:54.047751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:42179 (local address 127.7.67.131:41673)
0811 05:13:54.047945 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:54.047961 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:54.048039 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:54.048385 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:54.048389 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:54.048400 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 05:13:54.048567 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:54.048578 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:54.049730 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:54.049734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:54.050765 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:54.050775 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:54.051264 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:54.051268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:54.051363 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:54.051398 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 05:13:54.051698 (+ 300us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 05:13:54.051705 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 05:13:54.051775 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 05:13:54.051862 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 05:13:54.081162 19547 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:54.081323 19547 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:13:54.084007 19547 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c0c901ac6d7f4906a97d099e134ffe09: Sending DeleteTablet for 3 replicas of tablet 4f8815d2091e4d79ada65ae72c064e86
I20260811 05:13:54.084728 20009 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f8815d2091e4d79ada65ae72c064e86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:54 UTC) from {username='oryx', principal='oryx/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:45470
I20260811 05:13:54.084877 20238 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f8815d2091e4d79ada65ae72c064e86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:54 UTC) from {username='oryx', principal='oryx/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:57300
I20260811 05:13:54.084896 19764 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f8815d2091e4d79ada65ae72c064e86 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:13:54 UTC) from {username='oryx', principal='oryx/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:48188
I20260811 05:13:54.084946 20441 tablet_replica.cc:330] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903: stopping tablet replica
I20260811 05:13:54.085060 20439 tablet_replica.cc:330] T 4f8815d2091e4d79ada65ae72c064e86 P 73460f11a7024fc9843372b07ee57dae: stopping tablet replica
I20260811 05:13:54.085072 20441 raft_consensus.cc:2229] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:13:54.085175 20439 raft_consensus.cc:2229] T 4f8815d2091e4d79ada65ae72c064e86 P 73460f11a7024fc9843372b07ee57dae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:54.085314 20439 raft_consensus.cc:2258] T 4f8815d2091e4d79ada65ae72c064e86 P 73460f11a7024fc9843372b07ee57dae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:54.085404 20441 raft_consensus.cc:2258] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:54.085695 20440 tablet_replica.cc:330] T 4f8815d2091e4d79ada65ae72c064e86 P 945234845f614c6793619eee11183edc: stopping tablet replica
I20260811 05:13:54.085825 20440 raft_consensus.cc:2229] T 4f8815d2091e4d79ada65ae72c064e86 P 945234845f614c6793619eee11183edc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:13:54.085945 20440 raft_consensus.cc:2258] T 4f8815d2091e4d79ada65ae72c064e86 P 945234845f614c6793619eee11183edc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:13:54.086221 20439 ts_tablet_manager.cc:1905] T 4f8815d2091e4d79ada65ae72c064e86 P 73460f11a7024fc9843372b07ee57dae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:54.086220 20441 ts_tablet_manager.cc:1905] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:54.087148 20440 ts_tablet_manager.cc:1905] T 4f8815d2091e4d79ada65ae72c064e86 P 945234845f614c6793619eee11183edc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:13:54.088958 20439 ts_tablet_manager.cc:1918] T 4f8815d2091e4d79ada65ae72c064e86 P 73460f11a7024fc9843372b07ee57dae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:13:54.088958 20441 ts_tablet_manager.cc:1918] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:13:54.089040 20441 log.cc:1196] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/4f8815d2091e4d79ada65ae72c064e86
I20260811 05:13:54.089040 20439 log.cc:1196] T 4f8815d2091e4d79ada65ae72c064e86 P 73460f11a7024fc9843372b07ee57dae: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/4f8815d2091e4d79ada65ae72c064e86
I20260811 05:13:54.089331 20439 ts_tablet_manager.cc:1939] T 4f8815d2091e4d79ada65ae72c064e86 P 73460f11a7024fc9843372b07ee57dae: Deleting consensus metadata
I20260811 05:13:54.089331 20441 ts_tablet_manager.cc:1939] T 4f8815d2091e4d79ada65ae72c064e86 P 2906ab24d87b4e97abae1f364a25e903: Deleting consensus metadata
I20260811 05:13:54.089957 20440 ts_tablet_manager.cc:1918] T 4f8815d2091e4d79ada65ae72c064e86 P 945234845f614c6793619eee11183edc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 05:13:54.090044 20440 log.cc:1196] T 4f8815d2091e4d79ada65ae72c064e86 P 945234845f614c6793619eee11183edc: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786425190664461-7438-0/minicluster-data/ts-0/wal/wals/4f8815d2091e4d79ada65ae72c064e86
I20260811 05:13:54.090086 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 19659
I20260811 05:13:54.090199 19525 catalog_manager.cc:4841] TS 73460f11a7024fc9843372b07ee57dae (127.7.67.131:41747): tablet 4f8815d2091e4d79ada65ae72c064e86 (table test-table [id=7bed81586c6b45b099ea439db13c011e]) successfully deleted
I20260811 05:13:54.109455 19524 catalog_manager.cc:4841] TS 2906ab24d87b4e97abae1f364a25e903 (127.7.67.130:42179): tablet 4f8815d2091e4d79ada65ae72c064e86 (table test-table [id=7bed81586c6b45b099ea439db13c011e]) successfully deleted
I20260811 05:13:54.114557 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 19897
W20260811 05:13:54.114522 19523 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.129:46641 (error 108)
I20260811 05:13:54.120944 20544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:54.120481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:46641 (local address 127.0.0.1:48204)
0811 05:13:54.120767 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:54.120848 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.129:46641: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":125,"threads_started":1}
W20260811 05:13:54.121098 19523 catalog_manager.cc:4584] TS 945234845f614c6793619eee11183edc (127.7.67.129:46641): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4f8815d2091e4d79ada65ae72c064e86: Network error: Client connection negotiation failed: client connection to 127.7.67.129:46641: connect: Connection refused (error 111)
I20260811 05:13:54.127238 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 20125
I20260811 05:13:54.142549 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 19498
2026-08-11T05:13:54Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3244 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:13:54 dist-test-slave-04bb krb5kdc[20547](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:13:54 dist-test-slave-04bb krb5kdc[20547](info): set up 2 sockets
Aug 11 05:13:54 dist-test-slave-04bb krb5kdc[20547](info): commencing operation
krb5kdc: starting...
W20260811 05:13:55.442768 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:13:55 dist-test-slave-04bb krb5kdc[20547](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425235, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:13:55.571991 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:39589
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:35249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:39589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:55.687811 20570 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:55.688120 20570 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:55.688190 20570 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:55.692108 20570 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:55.692204 20570 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:55.692245 20570 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:55.692305 20570 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:55.692345 20570 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:55.697540 20570 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:35249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:39589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:39589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20570
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:55.698621 20570 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:55.699666 20570 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:55.708372 20581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:55.708361 20577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:55.708361 20578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:55.708670 20570 server_base.cc:1034] running on GCE node
I20260811 05:13:55.709261 20570 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:55.710304 20570 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:55.711483 20570 hybrid_clock.cc:648] HybridClock initialized: now 1786425235711462 us; error 33 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T05:13:55Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1570 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:13:55 dist-test-slave-04bb krb5kdc[20589](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:13:55 dist-test-slave-04bb krb5kdc[20589](info): set up 2 sockets
Aug 11 05:13:55 dist-test-slave-04bb krb5kdc[20589](info): commencing operation
krb5kdc: starting...
W20260811 05:13:56.979936 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:13:57 dist-test-slave-04bb krb5kdc[20589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:13:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:13:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:13:57.109911 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:37883
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:37883
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:57.225306 20607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:57.225621 20607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:57.225663 20607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:57.229655 20607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:57.229713 20607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:57.229733 20607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:57.229749 20607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:57.229784 20607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:57.234879 20607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:37883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:37883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:57.235970 20607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:57.237032 20607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:57.245417 20612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:57.245510 20607 server_base.cc:1034] running on GCE node
W20260811 05:13:57.245419 20617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:57.245419 20613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:57.246235 20607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:57.247206 20607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:57.248374 20607 hybrid_clock.cc:648] HybridClock initialized: now 1786425237248359 us; error 28 us; skew 500 ppm
Aug 11 05:13:57 dist-test-slave-04bb krb5kdc[20589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425237, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:57.251242 20607 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:13:57.252140 20607 webserver.cc:460] Webserver started at http://127.7.67.190:43533/ using document root <none> and password file <none>
I20260811 05:13:57.252766 20607 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:57.252830 20607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:57.253044 20607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:57.255043 20607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "50df3f9ab0dc432c84379cf6e2c90acc"
format_stamp: "Formatted at 2026-08-11 05:13:57 on dist-test-slave-04bb"
server_key: "f67393140278266683b0bfc1368e4bb9"
server_key_iv: "e2dc00ae482b2627f1295052d07d4857"
server_key_version: "encryptionkey@0"
I20260811 05:13:57.255600 20607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "50df3f9ab0dc432c84379cf6e2c90acc"
format_stamp: "Formatted at 2026-08-11 05:13:57 on dist-test-slave-04bb"
server_key: "f67393140278266683b0bfc1368e4bb9"
server_key_iv: "e2dc00ae482b2627f1295052d07d4857"
server_key_version: "encryptionkey@0"
I20260811 05:13:57.259321 20607 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 05:13:57.261803 20628 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:57.263101 20607 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:57.263290 20607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "50df3f9ab0dc432c84379cf6e2c90acc"
format_stamp: "Formatted at 2026-08-11 05:13:57 on dist-test-slave-04bb"
server_key: "f67393140278266683b0bfc1368e4bb9"
server_key_iv: "e2dc00ae482b2627f1295052d07d4857"
server_key_version: "encryptionkey@0"
I20260811 05:13:57.263410 20607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:57.281160 20607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:57.284232 20607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:57.284433 20607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:57.291814 20607 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:37883
I20260811 05:13:57.291834 20705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:37883 every 8 connection(s)
I20260811 05:13:57.292881 20607 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:57.295749 20706 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:13:57.296700 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 20607
I20260811 05:13:57.296840 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:13:57.297116 7438 external_mini_cluster.cc:1428] Setting key dc59b93e28520c4ca99a95eb1ca46193
I20260811 05:13:57.302646 20706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc: Bootstrap starting.
I20260811 05:13:57.305354 20706 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc: Neither blocks nor log segments found. Creating new log.
I20260811 05:13:57.306088 20706 log.cc:824] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc: Log is configured to *not* fsync() on all Append() calls
Aug 11 05:13:57 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425237, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:57.308586 20706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc: No bootstrap required, opened a new log
I20260811 05:13:57.312729 20709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:57.298566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36952 (local address 127.7.67.190:37883)
0811 05:13:57.298996 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:57.299017 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:57.299063 (+ 46us) server_negotiation.cc:407] Connection header received
0811 05:13:57.299906 (+ 843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:57.299932 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:57.300341 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:57.300739 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:57.301479 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:57.302941 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:57.303867 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:57.304516 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:57.304710 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:57.307454 (+ 2744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:57.307460 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:57.307477 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:57.307507 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:57.309525 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:57.309960 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:57.309966 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:57.309972 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:57.310116 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:57.310410 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:57.310413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:57.310415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:57.310982 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:57.311207 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:57.311447 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 05:13:57.311748 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":113,"threads_started":1}
I20260811 05:13:57.313242 20706 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50df3f9ab0dc432c84379cf6e2c90acc" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37883 } }
I20260811 05:13:57.313504 20706 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:13:57.313589 20706 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50df3f9ab0dc432c84379cf6e2c90acc, State: Initialized, Role: FOLLOWER
I20260811 05:13:57.314199 20706 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50df3f9ab0dc432c84379cf6e2c90acc" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37883 } }
I20260811 05:13:57.314337 20706 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:57.314427 20706 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:57.314543 20706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:13:57.315593 20706 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50df3f9ab0dc432c84379cf6e2c90acc" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37883 } }
I20260811 05:13:57.315961 20706 leader_election.cc:302] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50df3f9ab0dc432c84379cf6e2c90acc; no voters:
I20260811 05:13:57.316273 20706 leader_election.cc:288] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:13:57.316426 20713 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:13:57.316720 20713 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [term 1 LEADER]: Becoming Leader. State: Replica: 50df3f9ab0dc432c84379cf6e2c90acc, State: Running, Role: LEADER
I20260811 05:13:57.317039 20713 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50df3f9ab0dc432c84379cf6e2c90acc" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37883 } }
I20260811 05:13:57.317586 20706 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:57.318164 20713 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50df3f9ab0dc432c84379cf6e2c90acc. Latest consensus state: current_term: 1 leader_uuid: "50df3f9ab0dc432c84379cf6e2c90acc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50df3f9ab0dc432c84379cf6e2c90acc" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37883 } } }
I20260811 05:13:57.318300 20713 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:57.318732 20715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50df3f9ab0dc432c84379cf6e2c90acc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50df3f9ab0dc432c84379cf6e2c90acc" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37883 } } }
I20260811 05:13:57.318900 20715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:57.318949 20721 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:57.322062 20721 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:57.327515 20721 catalog_manager.cc:1347] Generated new cluster ID: 48731ec2ee4843cc80d71d9e486e491d
I20260811 05:13:57.327589 20721 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:57.335675 20721 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:13:57.336917 20721 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:57.350431 20721 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc: Generated new TSK 0
I20260811 05:13:57.351075 20721 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
I20260811 05:13:57.379930 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:37883
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:57.501046 20747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:57.501312 20747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:57.501356 20747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:57.505231 20747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:57.505285 20747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:57.505344 20747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:13:57.510494 20747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:37883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:57.511473 20747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:57.512459 20747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:57.522337 20757 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:57.522352 20763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:57.522639 20747 server_base.cc:1034] running on GCE node
W20260811 05:13:57.522405 20760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:57.523149 20747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:57.523820 20747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:57.525038 20747 hybrid_clock.cc:648] HybridClock initialized: now 1786425237524992 us; error 58 us; skew 500 ppm
Aug 11 05:13:57 dist-test-slave-04bb krb5kdc[20589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425237, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:57.528023 20747 init.cc:377] Logged in from keytab as kudu/127.7.67.129@KRBTEST.COM (short username kudu)
I20260811 05:13:57.529058 20747 webserver.cc:460] Webserver started at http://127.7.67.129:40809/ using document root <none> and password file <none>
I20260811 05:13:57.529688 20747 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:57.529788 20747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:57.530045 20747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:57.532238 20747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "eaaca759d0454f09978882c284a9e4e9"
format_stamp: "Formatted at 2026-08-11 05:13:57 on dist-test-slave-04bb"
server_key: "774739ca5ba81d91907488dfb2d15b09"
server_key_iv: "6a840cbcdc42772205aefa3a2219a92e"
server_key_version: "encryptionkey@0"
I20260811 05:13:57.532871 20747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "eaaca759d0454f09978882c284a9e4e9"
format_stamp: "Formatted at 2026-08-11 05:13:57 on dist-test-slave-04bb"
server_key: "774739ca5ba81d91907488dfb2d15b09"
server_key_iv: "6a840cbcdc42772205aefa3a2219a92e"
server_key_version: "encryptionkey@0"
I20260811 05:13:57.536963 20747 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 05:13:57.539541 20774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:57.540882 20747 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:13:57.541024 20747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "eaaca759d0454f09978882c284a9e4e9"
format_stamp: "Formatted at 2026-08-11 05:13:57 on dist-test-slave-04bb"
server_key: "774739ca5ba81d91907488dfb2d15b09"
server_key_iv: "6a840cbcdc42772205aefa3a2219a92e"
server_key_version: "encryptionkey@0"
I20260811 05:13:57.541143 20747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:57.585179 20747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:57.588495 20747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:57.588742 20747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:57.589406 20747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:57.590318 20747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:57.590396 20747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:57.590476 20747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:57.590533 20747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:57.600723 20747 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:39161
I20260811 05:13:57.600776 20951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:39161 every 8 connection(s)
I20260811 05:13:57.601660 20747 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:57.607534 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 20747
I20260811 05:13:57.607661 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:13:57.607990 7438 external_mini_cluster.cc:1428] Setting key 5d6d13e0718237bbba5ea2f598fb7123
Aug 11 05:13:57 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425237, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:57.616663 20709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:57.603497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:33147 (local address 127.7.67.190:37883)
0811 05:13:57.603654 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:57.603658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:57.604485 (+ 827us) server_negotiation.cc:407] Connection header received
0811 05:13:57.606042 (+ 1557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:57.606045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:57.606097 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:57.606215 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:57.607432 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:57.608249 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:57.609723 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:57.610127 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:57.610256 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:57.612847 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:57.612851 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:57.612854 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:57.612879 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:57.614382 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:57.614956 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:57.614960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:57.614961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:57.615043 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:57.615438 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:57.615442 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:57.615444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:57.615636 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:57.615759 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:57.616289 (+ 530us) server_negotiation.cc:299] Negotiation successful
0811 05:13:57.616426 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 05:13:57.617556 20957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:57.603854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.129:33147)
0811 05:13:57.604317 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:57.604363 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:57.605822 (+ 1459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:57.606386 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:57.606393 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:57.606759 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:57.607257 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:57.607271 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:57.608408 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:57.608412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:57.609596 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:57.609606 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:57.610281 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:57.610284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:57.610431 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:57.611061 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:57.611079 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:57.612657 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:57.614515 (+ 1858us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:57.614521 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:57.614536 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:57.614826 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:57.615147 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:57.615151 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:57.615153 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:57.615322 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:57.615761 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:57.615769 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:57.616021 (+ 252us) client_negotiation.cc:769] Sending connection context
0811 05:13:57.616216 (+ 195us) client_negotiation.cc:240] Negotiation successful
0811 05:13:57.616571 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":126,"threads_started":1}
WARNING: no policy specified for kudu/127.7.67.130@KRBTEST.COM; defaulting to no policy
I20260811 05:13:57.618927 20952 heartbeater.cc:347] Connected to a master server at 127.7.67.190:37883
I20260811 05:13:57.619185 20952 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:57.619751 20952 heartbeater.cc:511] Master 127.7.67.190:37883 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.130@KRBTEST.COM" created.
I20260811 05:13:57.621536 20654 ts_manager.cc:194] Registered new tserver with Master: eaaca759d0454f09978882c284a9e4e9 (127.7.67.129:39161)
I20260811 05:13:57.622758 20654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.129@KRBTEST.COM'} at 127.7.67.129:33147
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
I20260811 05:13:57.639110 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:37883
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:57.750955 20968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:57.751189 20968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:57.751230 20968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:57.755067 20968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:57.755116 20968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:57.755175 20968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:13:57.760041 20968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:37883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:57.760998 20968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:57.762032 20968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:57.771863 20978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:57.771863 20976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:57.771919 20974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:57.772305 20968 server_base.cc:1034] running on GCE node
I20260811 05:13:57.772835 20968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:57.773507 20968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:57.774689 20968 hybrid_clock.cc:648] HybridClock initialized: now 1786425237774669 us; error 35 us; skew 500 ppm
Aug 11 05:13:57 dist-test-slave-04bb krb5kdc[20589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425237, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:57.777729 20968 init.cc:377] Logged in from keytab as kudu/127.7.67.130@KRBTEST.COM (short username kudu)
I20260811 05:13:57.778623 20968 webserver.cc:460] Webserver started at http://127.7.67.130:38369/ using document root <none> and password file <none>
I20260811 05:13:57.779199 20968 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:57.779263 20968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:57.779505 20968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:57.781518 20968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "7b773287b392432aaa2fc84908e4cd20"
format_stamp: "Formatted at 2026-08-11 05:13:57 on dist-test-slave-04bb"
server_key: "922d0a5a83b576830591aa478ec7da9f"
server_key_iv: "dce35bf31cdf3d98f14722671f28d0bb"
server_key_version: "encryptionkey@0"
I20260811 05:13:57.782059 20968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "7b773287b392432aaa2fc84908e4cd20"
format_stamp: "Formatted at 2026-08-11 05:13:57 on dist-test-slave-04bb"
server_key: "922d0a5a83b576830591aa478ec7da9f"
server_key_iv: "dce35bf31cdf3d98f14722671f28d0bb"
server_key_version: "encryptionkey@0"
I20260811 05:13:57.785673 20968 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 05:13:57.788010 20989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:57.789311 20968 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:13:57.789451 20968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "7b773287b392432aaa2fc84908e4cd20"
format_stamp: "Formatted at 2026-08-11 05:13:57 on dist-test-slave-04bb"
server_key: "922d0a5a83b576830591aa478ec7da9f"
server_key_iv: "dce35bf31cdf3d98f14722671f28d0bb"
server_key_version: "encryptionkey@0"
I20260811 05:13:57.789561 20968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:57.813747 20968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:57.817092 20968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:57.817338 20968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:57.818034 20968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:57.819100 20968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:57.819176 20968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:57.819275 20968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:57.819320 20968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:57.828809 20968 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:37903
I20260811 05:13:57.828828 21159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:37903 every 8 connection(s)
I20260811 05:13:57.829700 20968 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:57.836359 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 20968
I20260811 05:13:57.836514 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:13:57.836827 7438 external_mini_cluster.cc:1428] Setting key b8072070a99f5ca92fbb806da4edf0b5
Aug 11 05:13:57 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425237, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:57.844285 20709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:57.831550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:45641 (local address 127.7.67.190:37883)
0811 05:13:57.831683 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:57.831687 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:57.832552 (+ 865us) server_negotiation.cc:407] Connection header received
0811 05:13:57.833854 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:57.833858 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:57.833915 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:57.834020 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:57.835240 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:57.835783 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:57.837098 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:57.837327 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:57.837437 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:57.840265 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:57.840268 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:57.840270 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:57.840293 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:57.841983 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:57.842639 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:57.842644 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:57.842646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:57.842740 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:57.843086 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:57.843090 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:57.843092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:57.843300 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:57.843414 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:57.843950 (+ 536us) server_negotiation.cc:299] Negotiation successful
0811 05:13:57.844081 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 05:13:57.845227 21163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:57.831942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.130:45641)
0811 05:13:57.832379 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:57.832422 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:57.833644 (+ 1222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:57.834202 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:57.834209 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:57.834572 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:57.835092 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:57.835105 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:57.835908 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:57.835911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:57.836977 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:57.836988 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:57.837481 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:57.837485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:57.837656 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:57.838247 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:57.838270 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:57.840048 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:57.842118 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:57.842125 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:57.842142 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:57.842532 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:57.842836 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:57.842840 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:57.842842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:57.842984 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:57.843412 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:57.843425 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:57.843691 (+ 266us) client_negotiation.cc:769] Sending connection context
0811 05:13:57.843903 (+ 212us) client_negotiation.cc:240] Negotiation successful
0811 05:13:57.844238 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":121,"threads_started":1}
I20260811 05:13:57.846465 21161 heartbeater.cc:347] Connected to a master server at 127.7.67.190:37883
I20260811 05:13:57.846755 21161 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:57.847383 21161 heartbeater.cc:511] Master 127.7.67.190:37883 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.67.131@KRBTEST.COM; defaulting to no policy
I20260811 05:13:57.848694 20654 ts_manager.cc:194] Registered new tserver with Master: 7b773287b392432aaa2fc84908e4cd20 (127.7.67.130:37903)
I20260811 05:13:57.849393 20654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.130@KRBTEST.COM'} at 127.7.67.130:45641
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
I20260811 05:13:57.867935 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:37883
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:57.981391 21168 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:57.981629 21168 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:57.981670 21168 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:57.985515 21168 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:57.985563 21168 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:57.985617 21168 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:13:57.990618 21168 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:37883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21168
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:57.991518 21168 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:57.992504 21168 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:58.001971 21178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:58.001971 21176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:58.002099 21173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:58.002476 21168 server_base.cc:1034] running on GCE node
I20260811 05:13:58.002897 21168 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:58.003538 21168 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:58.004755 21168 hybrid_clock.cc:648] HybridClock initialized: now 1786425238004710 us; error 56 us; skew 500 ppm
Aug 11 05:13:58 dist-test-slave-04bb krb5kdc[20589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425238, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:58.008011 21168 init.cc:377] Logged in from keytab as kudu/127.7.67.131@KRBTEST.COM (short username kudu)
I20260811 05:13:58.008965 21168 webserver.cc:460] Webserver started at http://127.7.67.131:37309/ using document root <none> and password file <none>
I20260811 05:13:58.009570 21168 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:58.009663 21168 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:58.009902 21168 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:13:58.012105 21168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "b646f01b4cf2486097fb7a1c76539339"
format_stamp: "Formatted at 2026-08-11 05:13:58 on dist-test-slave-04bb"
server_key: "3cda7e1910f1942a3ff9bc7ebd6dc90e"
server_key_iv: "b5c8f3122170a36acce80f53676827d1"
server_key_version: "encryptionkey@0"
I20260811 05:13:58.012779 21168 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "b646f01b4cf2486097fb7a1c76539339"
format_stamp: "Formatted at 2026-08-11 05:13:58 on dist-test-slave-04bb"
server_key: "3cda7e1910f1942a3ff9bc7ebd6dc90e"
server_key_iv: "b5c8f3122170a36acce80f53676827d1"
server_key_version: "encryptionkey@0"
I20260811 05:13:58.016510 21168 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:13:58.019603 21193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:58.021009 21168 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 05:13:58.021160 21168 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "b646f01b4cf2486097fb7a1c76539339"
format_stamp: "Formatted at 2026-08-11 05:13:58 on dist-test-slave-04bb"
server_key: "3cda7e1910f1942a3ff9bc7ebd6dc90e"
server_key_iv: "b5c8f3122170a36acce80f53676827d1"
server_key_version: "encryptionkey@0"
I20260811 05:13:58.021291 21168 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:58.042179 21168 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:58.045188 21168 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:58.045384 21168 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:58.045991 21168 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:58.046873 21168 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:58.046942 21168 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:58.047009 21168 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:58.047060 21168 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:58.055966 21168 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:39611
I20260811 05:13:58.056006 21370 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:39611 every 8 connection(s)
I20260811 05:13:58.056947 21168 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:13:58.064961 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 21168
I20260811 05:13:58.065110 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:13:58.065460 7438 external_mini_cluster.cc:1428] Setting key 16f054333adbbe0015d396549747e324
Aug 11 05:13:58 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425238, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:13:58.072343 20709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:58.058789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:36509 (local address 127.7.67.190:37883)
0811 05:13:58.058947 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:58.058953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:58.059819 (+ 866us) server_negotiation.cc:407] Connection header received
0811 05:13:58.061174 (+ 1355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:58.061177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:58.061229 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:58.061308 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:58.062640 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.063173 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.064333 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.064672 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.064785 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:58.068019 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:58.068023 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:58.068025 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:58.068051 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:58.069944 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:58.070583 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:58.070592 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:58.070594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:58.070688 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:58.071096 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:58.071102 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:58.071104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:58.071319 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:58.071444 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:58.072011 (+ 567us) server_negotiation.cc:299] Negotiation successful
0811 05:13:58.072163 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 05:13:58.073315 21374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:58.059204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.131:36509)
0811 05:13:58.059654 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:58.059697 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:58.060968 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:58.061540 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:58.061547 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:58.061951 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:58.062472 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:58.062488 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.063312 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.063315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:58.064188 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:58.064197 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.064865 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.064870 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:58.065091 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:58.065830 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:58.065853 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:58.067811 (+ 1958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:13:58.070092 (+ 2281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:58.070099 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:58.070119 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:58.070452 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:58.070797 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:13:58.070801 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:13:58.070804 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:13:58.070978 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:13:58.071441 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:13:58.071449 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:13:58.071745 (+ 296us) client_negotiation.cc:769] Sending connection context
0811 05:13:58.071965 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 05:13:58.072353 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":120,"threads_started":1}
I20260811 05:13:58.074576 21371 heartbeater.cc:347] Connected to a master server at 127.7.67.190:37883
I20260811 05:13:58.074885 21371 heartbeater.cc:464] Registering TS with master...
I20260811 05:13:58.075426 21371 heartbeater.cc:511] Master 127.7.67.190:37883 requested a full tablet report, sending...
I20260811 05:13:58.076639 20654 ts_manager.cc:194] Registered new tserver with Master: b646f01b4cf2486097fb7a1c76539339 (127.7.67.131:39611)
I20260811 05:13:58.077231 20654 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.131@KRBTEST.COM'} at 127.7.67.131:36509
I20260811 05:13:58.079798 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.7.67.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.129.keytab.
Entry for principal oryx/127.7.67.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.129.keytab.
WARNING: no policy specified for oryx/127.7.67.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.130.keytab.
Entry for principal oryx/127.7.67.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.130.keytab.
WARNING: no policy specified for oryx/127.7.67.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.7.67.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.7.67.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.131.keytab.
Entry for principal oryx/127.7.67.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.131.keytab.
I20260811 05:13:58.161641 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 20747
I20260811 05:13:58.181869 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 20968
I20260811 05:13:58.191900 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 21168
I20260811 05:13:58.202986 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 20607
I20260811 05:13:58.212355 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:37883
--webserver_interface=127.7.67.190
--webserver_port=43533
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:37883
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:13:58.331302 21385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:58.331542 21385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:58.331580 21385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:58.335417 21385 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:13:58.335469 21385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:58.335489 21385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:58.335505 21385 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:13:58.335520 21385 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:13:58.340613 21385 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:37883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:37883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=43533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:58.341668 21385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:58.342682 21385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:13:58.351182 21385 server_base.cc:1034] running on GCE node
W20260811 05:13:58.351114 21392 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:58.351125 21393 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:58.351101 21395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:58.352025 21385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:58.353163 21385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:58.354375 21385 hybrid_clock.cc:648] HybridClock initialized: now 1786425238354358 us; error 29 us; skew 500 ppm
Aug 11 05:13:58 dist-test-slave-04bb krb5kdc[20589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425238, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:58.357163 21385 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:13:58.358095 21385 webserver.cc:460] Webserver started at http://127.7.67.190:43533/ using document root <none> and password file <none>
I20260811 05:13:58.358666 21385 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:58.358783 21385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:58.362720 21385 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 05:13:58.364923 21407 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:58.366086 21385 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:13:58.366228 21385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "50df3f9ab0dc432c84379cf6e2c90acc"
format_stamp: "Formatted at 2026-08-11 05:13:57 on dist-test-slave-04bb"
server_key: "f67393140278266683b0bfc1368e4bb9"
server_key_iv: "e2dc00ae482b2627f1295052d07d4857"
server_key_version: "encryptionkey@0"
I20260811 05:13:58.366776 21385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:58.386840 21385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:58.389727 21385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:58.389935 21385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:58.397233 21385 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:37883
I20260811 05:13:58.397261 21483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:37883 every 8 connection(s)
I20260811 05:13:58.398329 21385 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:13:58.399091 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 21385
I20260811 05:13:58.399598 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:39161
--local_ip_for_outbound_sockets=127.7.67.129
--tserver_master_addrs=127.7.67.190:37883
--webserver_port=40809
--webserver_interface=127.7.67.129
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.67.129
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.129.keytab with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:13:58.403698 21484 sys_catalog.cc:263] Verifying existing consensus state
I20260811 05:13:58.405521 21484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc: Bootstrap starting.
I20260811 05:13:58.422976 21484 log.cc:824] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc: Log is configured to *not* fsync() on all Append() calls
I20260811 05:13:58.429947 21484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:13:58.430343 21484 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc: Bootstrap complete.
I20260811 05:13:58.434772 21484 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50df3f9ab0dc432c84379cf6e2c90acc" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37883 } }
I20260811 05:13:58.435375 21484 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50df3f9ab0dc432c84379cf6e2c90acc, State: Initialized, Role: FOLLOWER
I20260811 05:13:58.435956 21484 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50df3f9ab0dc432c84379cf6e2c90acc" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37883 } }
I20260811 05:13:58.436112 21484 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:13:58.436208 21484 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:13:58.436313 21484 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [term 1 FOLLOWER]: Advancing to term 2
I20260811 05:13:58.438901 21484 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50df3f9ab0dc432c84379cf6e2c90acc" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37883 } }
I20260811 05:13:58.439258 21484 leader_election.cc:302] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50df3f9ab0dc432c84379cf6e2c90acc; no voters:
I20260811 05:13:58.439618 21484 leader_election.cc:288] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 05:13:58.439718 21490 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [term 2 FOLLOWER]: Leader election won for term 2
I20260811 05:13:58.439936 21490 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [term 2 LEADER]: Becoming Leader. State: Replica: 50df3f9ab0dc432c84379cf6e2c90acc, State: Running, Role: LEADER
I20260811 05:13:58.440233 21490 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50df3f9ab0dc432c84379cf6e2c90acc" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37883 } }
I20260811 05:13:58.440918 21484 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:13:58.441231 21490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50df3f9ab0dc432c84379cf6e2c90acc. Latest consensus state: current_term: 2 leader_uuid: "50df3f9ab0dc432c84379cf6e2c90acc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50df3f9ab0dc432c84379cf6e2c90acc" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37883 } } }
I20260811 05:13:58.441358 21490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:58.442054 21492 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "50df3f9ab0dc432c84379cf6e2c90acc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50df3f9ab0dc432c84379cf6e2c90acc" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37883 } } }
I20260811 05:13:58.442157 21492 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc [sys.catalog]: This master's current role is: LEADER
I20260811 05:13:58.446262 21514 catalog_manager.cc:1259] Loaded cluster ID: 48731ec2ee4843cc80d71d9e486e491d
I20260811 05:13:58.446321 21514 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc: loading cluster ID for follower catalog manager: success
I20260811 05:13:58.448788 21514 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc: acquiring CA information for follower catalog manager: success
I20260811 05:13:58.450307 21514 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 05:13:58.450433 21496 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:13:58.451296 21496 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:13:58.451749 21496 catalog_manager.cc:1259] Loaded cluster ID: 48731ec2ee4843cc80d71d9e486e491d
I20260811 05:13:58.451812 21496 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:13:58.452430 21496 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:13:58.452927 21496 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 50df3f9ab0dc432c84379cf6e2c90acc: Loaded TSK: 0
I20260811 05:13:58.453696 21496 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 05:13:58.519593 21486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:58.519876 21486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:58.519951 21486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:58.523967 21486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:58.524057 21486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:58.524148 21486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:13:58.529628 21486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.129.keytab
--principal=oryx/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:39161
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=40809
--tserver_master_addrs=127.7.67.190:37883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:58.531008 21486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:58.532033 21486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:58.543140 21521 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:58.543151 21527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:58.543138 21522 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:58.544023 21486 server_base.cc:1034] running on GCE node
I20260811 05:13:58.544486 21486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:58.545208 21486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:58.546398 21486 hybrid_clock.cc:648] HybridClock initialized: now 1786425238546374 us; error 43 us; skew 500 ppm
Aug 11 05:13:58 dist-test-slave-04bb krb5kdc[20589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425238, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:58.549523 21486 init.cc:377] Logged in from keytab as oryx/127.7.67.129@KRBTEST.COM (short username oryx)
I20260811 05:13:58.550453 21486 webserver.cc:460] Webserver started at http://127.7.67.129:40809/ using document root <none> and password file <none>
I20260811 05:13:58.551028 21486 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:58.551143 21486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:58.555132 21486 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 05:13:58.557334 21538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:58.558652 21486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 05:13:58.558753 21486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "eaaca759d0454f09978882c284a9e4e9"
format_stamp: "Formatted at 2026-08-11 05:13:57 on dist-test-slave-04bb"
server_key: "774739ca5ba81d91907488dfb2d15b09"
server_key_iv: "6a840cbcdc42772205aefa3a2219a92e"
server_key_version: "encryptionkey@0"
I20260811 05:13:58.559295 21486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:58.571429 21486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:58.574679 21486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:58.574882 21486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:58.575521 21486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:58.576370 21486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:58.576442 21486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:58.576508 21486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:58.576588 21486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:58.587718 21486 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:39161
I20260811 05:13:58.587762 21713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:39161 every 8 connection(s)
I20260811 05:13:58.588780 21486 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:13:58.597623 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 21486
I20260811 05:13:58.598138 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:37903
--local_ip_for_outbound_sockets=127.7.67.130
--tserver_master_addrs=127.7.67.190:37883
--webserver_port=38369
--webserver_interface=127.7.67.130
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.67.130
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.130.keytab with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:13:58 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:58 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:58.601823 21717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:58.590995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.129:34377)
0811 05:13:58.591487 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:58.591540 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:58.592869 (+ 1329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:58.593912 (+ 1043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:58.593918 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:58.594277 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:58.594757 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:58.594770 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.595954 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.595956 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:58.596895 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:58.596905 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.597827 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.597830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:58.597965 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:58.598616 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:58.598632 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:58.601123 (+ 2491us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":153,"threads_started":1}
W20260811 05:13:58.602114 21716 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:58.590869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:34377 (local address 127.7.67.190:37883)
0811 05:13:58.591315 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:58.591341 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:58.591659 (+ 318us) server_negotiation.cc:407] Connection header received
0811 05:13:58.593232 (+ 1573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:58.593244 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:58.593528 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:58.593809 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:58.594915 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.595838 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.597009 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.597628 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.597821 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:58.601409 (+ 3588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.129:34377: BlockingRecv error: recv got EOF from 127.7.67.129:34377 (error 108)
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":146,"threads_started":1}
W20260811 05:13:58.602370 21714 heartbeater.cc:644] Failed to heartbeat to 127.7.67.190:37883 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.67.190:37883: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Aug 11 05:13:58 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:58 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:58.608804 21717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:58.602833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.129:47697)
0811 05:13:58.603015 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:58.603032 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:58.603118 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:58.603378 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:58.603384 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:58.603566 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:58.603702 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:58.603713 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.604487 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.604494 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:58.605550 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:58.605564 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.606001 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.606007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:58.606133 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:58.606653 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:58.606673 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:58.608657 (+ 1984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":100}
W20260811 05:13:58.608893 21716 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:58.602911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:47697 (local address 127.7.67.190:37883)
0811 05:13:58.603021 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:58.603024 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:58.603087 (+ 63us) server_negotiation.cc:407] Connection header received
0811 05:13:58.603223 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:58.603226 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:58.603265 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:58.603333 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:58.603819 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.604362 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.605680 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.605880 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.605960 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:58.608776 (+ 2816us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.129:47697: BlockingRecv error: recv got EOF from 127.7.67.129:47697 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 11 05:13:58 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:58 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:58.614807 21717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:58.609604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.129:52993)
0811 05:13:58.609856 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:58.609869 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:58.609976 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:58.610258 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:58.610262 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:58.610433 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:58.610502 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:58.610508 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.611231 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.611238 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:58.611754 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:58.611762 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.612246 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.612250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:58.612333 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:58.612885 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:58.612909 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:58.614661 (+ 1752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":71}
W20260811 05:13:58.614899 21716 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:58.609709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:52993 (local address 127.7.67.190:37883)
0811 05:13:58.609825 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:58.609828 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:58.609935 (+ 107us) server_negotiation.cc:407] Connection header received
0811 05:13:58.610086 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:58.610090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:58.610146 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:58.610242 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:58.610632 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.611112 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.611908 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.612125 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.612246 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:58.614778 (+ 2532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.129:52993: BlockingRecv error: recv got EOF from 127.7.67.129:52993 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260811 05:13:58.615221 21714 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 05:13:58.715349 21719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:58.715590 21719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:58.715633 21719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:58.719601 21719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:58.719655 21719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:58.719715 21719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:13:58.724938 21719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.130.keytab
--principal=oryx/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:37903
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=38369
--tserver_master_addrs=127.7.67.190:37883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:58.725955 21719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:58.726975 21719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:58.736809 21724 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:58.736887 21725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:58.736986 21719 server_base.cc:1034] running on GCE node
W20260811 05:13:58.736809 21728 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:58.737627 21719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:58.738507 21719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:58.739691 21719 hybrid_clock.cc:648] HybridClock initialized: now 1786425238739667 us; error 26 us; skew 500 ppm
Aug 11 05:13:58 dist-test-slave-04bb krb5kdc[20589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425238, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:58.742693 21719 init.cc:377] Logged in from keytab as oryx/127.7.67.130@KRBTEST.COM (short username oryx)
I20260811 05:13:58.743629 21719 webserver.cc:460] Webserver started at http://127.7.67.130:38369/ using document root <none> and password file <none>
I20260811 05:13:58.744194 21719 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:58.744270 21719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:58.748113 21719 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:13:58.750288 21736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:58.751497 21719 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:58.751638 21719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "7b773287b392432aaa2fc84908e4cd20"
format_stamp: "Formatted at 2026-08-11 05:13:57 on dist-test-slave-04bb"
server_key: "922d0a5a83b576830591aa478ec7da9f"
server_key_iv: "dce35bf31cdf3d98f14722671f28d0bb"
server_key_version: "encryptionkey@0"
I20260811 05:13:58.752170 21719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:58.803038 21719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:58.806449 21719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:58.806685 21719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:58.807394 21719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:58.808321 21719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:58.808394 21719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:58.808467 21719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:58.808511 21719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:58.818429 21719 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:37903
I20260811 05:13:58.818454 21911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:37903 every 8 connection(s)
I20260811 05:13:58.819478 21719 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:13:58.828056 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 21719
I20260811 05:13:58.828681 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:39611
--local_ip_for_outbound_sockets=127.7.67.131
--tserver_master_addrs=127.7.67.190:37883
--webserver_port=37309
--webserver_interface=127.7.67.131
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:33281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.7.67.131
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.131.keytab with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:13:58 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:58 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 05:13:58.831514 21716 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:58.821437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:46997 (local address 127.7.67.190:37883)
0811 05:13:58.821634 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:58.821638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:58.822501 (+ 863us) server_negotiation.cc:407] Connection header received
0811 05:13:58.823897 (+ 1396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:58.823900 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:58.823964 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:58.824075 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:58.825441 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.825972 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.827127 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.827353 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.827472 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:58.831371 (+ 3899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.130:46997: BlockingRecv error: recv got EOF from 127.7.67.130:46997 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
I20260811 05:13:58.832162 21915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:58.821830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.130:46997)
0811 05:13:58.822323 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:58.822368 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:58.823659 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:58.824273 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:58.824281 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:58.824665 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:58.825267 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:58.825283 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.826107 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.826113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:58.826992 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:58.827001 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.827520 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.827524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:58.827696 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:58.828395 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:58.828411 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:58.831230 (+ 2819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":115,"threads_started":1}
W20260811 05:13:58.832692 21912 heartbeater.cc:644] Failed to heartbeat to 127.7.67.190:37883 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.67.190:37883: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Aug 11 05:13:58 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:58 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:58.838707 21915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:58.833224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.130:41113)
0811 05:13:58.833363 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:58.833379 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:58.833470 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:58.833762 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:58.833765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:58.833936 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:58.834030 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:58.834035 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.834817 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.834822 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:58.835377 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:58.835385 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.835861 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.835865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:58.835960 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:58.836481 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:58.836493 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:58.838548 (+ 2055us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 05:13:58.838780 21716 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:58.833329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:41113 (local address 127.7.67.190:37883)
0811 05:13:58.833449 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:58.833453 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:58.833467 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:13:58.833601 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:58.833604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:58.833655 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:58.833746 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:58.834153 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.834677 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.835505 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.835734 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.835847 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:58.838667 (+ 2820us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.130:41113: BlockingRecv error: recv got EOF from 127.7.67.130:41113 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 05:13:58 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:58 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:58.845010 21915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:58.839407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.130:49123)
0811 05:13:58.839510 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:58.839524 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:58.839628 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:58.839931 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:58.839933 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:58.840055 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:58.840125 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:58.840129 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.841284 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.841287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:58.841946 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:58.841955 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.842525 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.842528 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:58.842584 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:58.843129 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:58.843141 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:58.844924 (+ 1783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 05:13:58.845189 21716 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:58.839518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:49123 (local address 127.7.67.190:37883)
0811 05:13:58.839681 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:58.839686 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:58.839702 (+ 16us) server_negotiation.cc:407] Connection header received
0811 05:13:58.839747 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:58.839752 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:58.839807 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:58.839898 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:58.840248 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.841164 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.842052 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:58.842426 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:58.842525 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:58.845069 (+ 2544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.130:49123: BlockingRecv error: recv got EOF from 127.7.67.130:49123 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 05:13:58.845467 21912 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 05:13:58.947430 21917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:13:58.947669 21917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:13:58.947762 21917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:13:58.951829 21917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:13:58.951881 21917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:13:58.951979 21917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:13:58.957300 21917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:33281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/krb5kdc/oryx_127.7.67.131.keytab
--principal=oryx/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:39611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=37309
--tserver_master_addrs=127.7.67.190:37883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:13:58.958304 21917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:13:58.959312 21917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:13:58.968950 21924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:58.968914 21927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:13:58.968976 21923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:13:58.969041 21917 server_base.cc:1034] running on GCE node
I20260811 05:13:58.969710 21917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:13:58.970359 21917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:13:58.971547 21917 hybrid_clock.cc:648] HybridClock initialized: now 1786425238971534 us; error 46 us; skew 500 ppm
Aug 11 05:13:58 dist-test-slave-04bb krb5kdc[20589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425238, etypes {rep=17 tkt=17 ses=17}, oryx/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:13:58.974407 21917 init.cc:377] Logged in from keytab as oryx/127.7.67.131@KRBTEST.COM (short username oryx)
I20260811 05:13:58.975258 21917 webserver.cc:460] Webserver started at http://127.7.67.131:37309/ using document root <none> and password file <none>
I20260811 05:13:58.975852 21917 fs_manager.cc:359] Metadata directory not provided
I20260811 05:13:58.975960 21917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:13:58.979861 21917 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:13:58.982023 21936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:58.983323 21917 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:13:58.983481 21917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "b646f01b4cf2486097fb7a1c76539339"
format_stamp: "Formatted at 2026-08-11 05:13:58 on dist-test-slave-04bb"
server_key: "3cda7e1910f1942a3ff9bc7ebd6dc90e"
server_key_iv: "b5c8f3122170a36acce80f53676827d1"
server_key_version: "encryptionkey@0"
I20260811 05:13:58.984019 21917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:13:58.996613 21917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:13:58.999408 21917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:13:58.999609 21917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:13:59.000193 21917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:13:59.001065 21917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:13:59.001139 21917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:59.001217 21917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:13:59.001261 21917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:13:59.010726 21917 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:39611
I20260811 05:13:59.010758 22110 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:39611 every 8 connection(s)
I20260811 05:13:59.011749 21917 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:13:59.016729 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 21917
Aug 11 05:13:59 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:59 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 05:13:59.023828 21716 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:59.013599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:53737 (local address 127.7.67.190:37883)
0811 05:13:59.013743 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:59.013747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:59.014498 (+ 751us) server_negotiation.cc:407] Connection header received
0811 05:13:59.015829 (+ 1331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:59.015832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:59.015904 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:59.015998 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:59.017405 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:59.018104 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:59.019712 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:59.020054 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:59.020127 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:59.023707 (+ 3580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.131:53737: BlockingRecv error: recv got EOF from 127.7.67.131:53737 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 05:13:59.024430 22113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:59.013901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.131:53737)
0811 05:13:59.014349 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:59.014390 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:59.015593 (+ 1203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:59.016178 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:59.016184 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:59.016626 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:59.017213 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:59.017238 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:59.018286 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:59.018288 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:59.019599 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:59.019608 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:59.020198 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:59.020201 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:59.020393 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:59.021018 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:59.021043 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:59.023594 (+ 2551us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":122,"threads_started":1}
W20260811 05:13:59.024964 22111 heartbeater.cc:644] Failed to heartbeat to 127.7.67.190:37883 (0 consecutive failures): Not authorized: Failed to ping master at 127.7.67.190:37883: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
I20260811 05:13:59.028467 22115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:59.017894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36966 (local address 127.7.67.190:37883)
0811 05:13:59.018163 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:59.018167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:59.018180 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:13:59.018216 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:59.018220 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:59.018272 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:59.018354 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:59.019319 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:59.019915 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:59.021122 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:59.021369 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:59.021447 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:59.023203 (+ 1756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:13:59.023209 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:13:59.023233 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:13:59.023262 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:13:59.026054 (+ 2792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:59.026598 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:59.026603 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:59.026608 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:59.026730 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:13:59.027068 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:13:59.027071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:13:59.027072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:13:59.027572 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:13:59.027775 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 05:13:59.028045 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 05:13:59.028307 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":80,"threads_started":1}
Aug 11 05:13:59 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:59 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:59.031525 22113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:59.025461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.131:39625)
0811 05:13:59.025585 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:59.025600 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:59.025690 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:59.025928 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:59.025933 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:59.026092 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:59.026213 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:59.026219 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:59.027234 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:59.027237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:59.028053 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:59.028064 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:59.028746 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:59.028748 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:59.028825 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:59.029344 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:59.029354 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:59.031371 (+ 2017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 05:13:59.031630 21716 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:59.025514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:39625 (local address 127.7.67.190:37883)
0811 05:13:59.025636 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:59.025640 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:59.025653 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:13:59.025778 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:59.025780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:59.025814 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:59.025882 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:59.026329 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:59.027116 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:59.028192 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:59.028493 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:59.028597 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:59.031518 (+ 2921us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.131:39625: BlockingRecv error: recv got EOF from 127.7.67.131:39625 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 05:13:59 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:59 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:59.037065 22113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:59.032087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.131:47105)
0811 05:13:59.032218 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:59.032235 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:59.032313 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:59.032615 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:59.032617 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:59.032753 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:59.032819 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:59.032824 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:59.033524 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:59.033528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:59.034106 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:59.034116 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:59.034559 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:59.034561 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:59.034623 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:59.035141 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:59.035154 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:59.036978 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 05:13:59.037194 21716 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:59.032210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:47105 (local address 127.7.67.190:37883)
0811 05:13:59.032361 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:59.032364 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:59.032376 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:13:59.032404 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:59.032406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:59.032436 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:59.032501 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:59.032924 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:59.033404 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:59.034257 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:59.034457 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:59.034532 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:59.037092 (+ 2560us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.131:47105: BlockingRecv error: recv got EOF from 127.7.67.131:47105 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260811 05:13:59.037357 22111 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 05:13:59 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:59 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:59.627084 22116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:59.615809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.129:39401)
0811 05:13:59.616280 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:59.616295 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:59.616398 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:59.616704 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:59.616706 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:59.616883 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:59.617001 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:59.617007 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:59.617785 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:59.617788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:59.618373 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:59.618382 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:59.618850 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:59.618852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:59.618922 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:59.624910 (+ 5988us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:59.624923 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:59.626967 (+ 2044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":100,"threads_started":1}
W20260811 05:13:59.627269 22117 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:59.615934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:39401 (local address 127.7.67.190:37883)
0811 05:13:59.616309 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:59.616313 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:59.616353 (+ 40us) server_negotiation.cc:407] Connection header received
0811 05:13:59.616479 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:59.616481 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:59.616581 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:59.616661 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:59.617109 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:59.617674 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:59.618496 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:59.618746 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:59.618830 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:59.627109 (+ 8279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.129:39401: BlockingRecv error: recv got EOF from 127.7.67.129:39401 (error 108)
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":103,"threads_started":1}
Aug 11 05:13:59 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:13:59 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:13:59.851265 22119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:13:59.846039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.130:47407)
0811 05:13:59.846312 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 05:13:59.846325 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:13:59.846421 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:13:59.846639 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:13:59.846641 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:13:59.846818 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:13:59.846936 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:59.846941 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:59.847677 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:59.847680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:59.848268 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:13:59.848276 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:13:59.848722 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:13:59.848725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:13:59.848797 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:59.849419 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:13:59.849431 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:13:59.851131 (+ 1700us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":96,"threads_started":1}
W20260811 05:13:59.851388 22117 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:13:59.846120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:47407 (local address 127.7.67.190:37883)
0811 05:13:59.846270 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 05:13:59.846274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:13:59.846375 (+ 101us) server_negotiation.cc:407] Connection header received
0811 05:13:59.846486 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:13:59.846488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:13:59.846532 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:13:59.846599 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:13:59.847044 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:59.847580 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:59.848371 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:13:59.848622 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:13:59.848701 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:13:59.851270 (+ 2569us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.130:47407: BlockingRecv error: recv got EOF from 127.7.67.130:47407 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 05:14:00 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:00 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:00.043974 22122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:00.037960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.131:47495)
0811 05:14:00.038251 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:00.038273 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:00.038369 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:00.038594 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:00.038596 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:00.038793 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:00.038888 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:00.038894 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:00.039648 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:00.039658 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:00.040527 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:00.040566 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:00.041183 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:00.041187 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:00.041285 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:00.041854 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:00.041865 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:00.043852 (+ 1987us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":119,"threads_started":1}
W20260811 05:14:00.044107 22117 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:00.038057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:47495 (local address 127.7.67.190:37883)
0811 05:14:00.038196 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:00.038200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:00.038341 (+ 141us) server_negotiation.cc:407] Connection header received
0811 05:14:00.038436 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:00.038439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:00.038485 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:00.038556 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:00.038989 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:00.039518 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:00.040692 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:00.041039 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:00.041139 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:00.043989 (+ 2850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.131:47495: BlockingRecv error: recv got EOF from 127.7.67.131:47495 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 05:14:00 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:00 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:00.634002 22126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:00.627874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.129:39889)
0811 05:14:00.628253 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:00.628270 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:00.628399 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:00.628688 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:00.628691 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:00.628902 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:00.629095 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:00.629107 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:00.629924 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:00.629927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:00.630502 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:00.630511 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:00.630979 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:00.630982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:00.631048 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:00.631579 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:00.631591 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:00.633869 (+ 2278us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":91,"threads_started":1}
W20260811 05:14:00.634199 22127 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:00.627988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:39889 (local address 127.7.67.190:37883)
0811 05:14:00.628271 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:00.628275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:00.628355 (+ 80us) server_negotiation.cc:407] Connection header received
0811 05:14:00.628487 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:00.628489 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:00.628554 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:00.628634 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:00.629261 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:00.629797 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:00.630619 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:00.630854 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:00.630938 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:00.634011 (+ 3073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.129:39889: BlockingRecv error: recv got EOF from 127.7.67.129:39889 (error 108)
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":91,"threads_started":1}
Aug 11 05:14:00 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:00 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:00.858321 22129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:00.852164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.130:48953)
0811 05:14:00.852504 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:00.852522 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:00.852690 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:00.853016 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:00.853020 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:00.853266 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:00.853376 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:00.853383 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:00.854238 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:00.854243 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:00.854882 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:00.854891 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:00.855427 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:00.855432 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:00.855528 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:00.856202 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:00.856215 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:00.858189 (+ 1974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":107,"threads_started":1}
W20260811 05:14:00.858469 22127 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:00.852338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:48953 (local address 127.7.67.190:37883)
0811 05:14:00.852506 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:00.852513 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:00.852640 (+ 127us) server_negotiation.cc:407] Connection header received
0811 05:14:00.852828 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:00.852837 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:00.852888 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:00.852984 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:00.853511 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:00.854112 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:00.855028 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:00.855285 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:00.855408 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:00.858347 (+ 2939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.130:48953: BlockingRecv error: recv got EOF from 127.7.67.130:48953 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 05:14:01 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:01 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:01.050868 22134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:01.044731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.131:50709)
0811 05:14:01.045037 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:01.045052 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:01.045156 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:01.045470 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:01.045472 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:01.045670 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:01.045778 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:01.045785 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:01.046531 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:01.046533 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:01.047146 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:01.047154 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:01.047580 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:01.047583 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:01.047649 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:01.048302 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:01.048315 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:01.050667 (+ 2352us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":102,"threads_started":1}
W20260811 05:14:01.050940 22127 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:01.044909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:50709 (local address 127.7.67.190:37883)
0811 05:14:01.045049 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:01.045052 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:01.045124 (+ 72us) server_negotiation.cc:407] Connection header received
0811 05:14:01.045286 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:01.045289 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:01.045342 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:01.045424 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:01.045891 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:01.046421 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:01.047258 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:01.047476 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:01.047553 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:01.050823 (+ 3270us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.131:50709: BlockingRecv error: recv got EOF from 127.7.67.131:50709 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 05:14:01 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:01 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:01.640195 22135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:01.634696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.129:57197)
0811 05:14:01.635000 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:01.635022 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:01.635113 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:01.635367 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:01.635369 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:01.635545 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:01.635643 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:01.635648 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:01.636407 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:01.636409 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:01.637020 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:01.637029 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:01.637442 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:01.637444 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:01.637502 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:01.637974 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:01.637985 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:01.640008 (+ 2023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":93,"threads_started":1}
W20260811 05:14:01.640295 22137 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:01.634792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:57197 (local address 127.7.67.190:37883)
0811 05:14:01.635100 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:01.635104 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:01.635117 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:14:01.635193 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:01.635195 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:01.635239 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:01.635317 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:01.635761 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:01.636308 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:01.637125 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:01.637341 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:01.637419 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:01.640150 (+ 2731us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.129:57197: BlockingRecv error: recv got EOF from 127.7.67.129:57197 (error 108)
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":108,"threads_started":1}
Aug 11 05:14:01 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:01 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:01.865557 22138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:01.859173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.130:59055)
0811 05:14:01.859584 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:01.859607 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:01.859739 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:01.860088 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:01.860091 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:01.860321 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:01.860460 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:01.860467 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:01.861502 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:01.861513 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:01.862099 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:01.862115 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:01.862598 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:01.862602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:01.862697 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:01.863323 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:01.863342 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:01.865429 (+ 2087us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":186,"threads_started":1}
W20260811 05:14:01.865814 22137 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:01.859382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:59055 (local address 127.7.67.190:37883)
0811 05:14:01.859524 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:01.859531 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:01.859701 (+ 170us) server_negotiation.cc:407] Connection header received
0811 05:14:01.859873 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:01.859877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:01.859940 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:01.860052 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:01.860597 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:01.861367 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:01.862229 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:01.862460 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:01.862545 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:01.865580 (+ 3035us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.130:59055: BlockingRecv error: recv got EOF from 127.7.67.130:59055 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 11 05:14:02 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:02 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:02.057731 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:02.051937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.131:42051)
0811 05:14:02.052245 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:02.052259 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:02.052363 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:02.052614 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:02.052616 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:02.052807 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:02.052901 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:02.052907 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:02.053608 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:02.053611 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:02.054172 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:02.054209 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:02.054622 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:02.054625 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:02.054692 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:02.055233 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:02.055245 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:02.057554 (+ 2309us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":99,"threads_started":1}
W20260811 05:14:02.057796 22137 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:02.051976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:42051 (local address 127.7.67.190:37883)
0811 05:14:02.052126 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:02.052129 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:02.052332 (+ 203us) server_negotiation.cc:407] Connection header received
0811 05:14:02.052445 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:02.052447 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:02.052498 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:02.052577 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:02.053006 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:02.053514 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:02.054305 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:02.054519 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:02.054595 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:02.057680 (+ 3085us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.131:42051: BlockingRecv error: recv got EOF from 127.7.67.131:42051 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 05:14:02 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:02 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:02.646842 22141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:02.641091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.129:33799)
0811 05:14:02.641399 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:02.641413 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:02.641509 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:02.641801 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:02.641803 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:02.642068 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:02.642204 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:02.642211 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:02.643097 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:02.643099 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:02.643693 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:02.643703 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:02.644158 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:02.644161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:02.644230 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:02.644874 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:02.644885 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:02.646695 (+ 1810us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":97,"threads_started":1}
W20260811 05:14:02.647059 22142 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:02.641154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:33799 (local address 127.7.67.190:37883)
0811 05:14:02.641457 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:02.641461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:02.641477 (+ 16us) server_negotiation.cc:407] Connection header received
0811 05:14:02.641602 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:02.641604 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:02.641659 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:02.641794 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:02.642328 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:02.642991 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:02.643810 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:02.644046 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:02.644135 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:02.646915 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.129:33799: BlockingRecv error: recv got EOF from 127.7.67.129:33799 (error 108)
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":101,"threads_started":1}
Aug 11 05:14:02 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:02 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:02.872884 22144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:02.866585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.130:34417)
0811 05:14:02.866943 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:02.866958 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:02.867065 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:02.867304 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:02.867306 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:02.867488 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:02.867583 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:02.867587 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:02.868473 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:02.868475 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:02.869195 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:02.869206 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:02.869804 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:02.869808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:02.869874 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:02.870523 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:02.870535 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:02.872693 (+ 2158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":174,"threads_started":1}
W20260811 05:14:02.872938 22142 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:02.866673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:34417 (local address 127.7.67.190:37883)
0811 05:14:02.866937 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:02.866941 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:02.867037 (+ 96us) server_negotiation.cc:407] Connection header received
0811 05:14:02.867158 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:02.867162 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:02.867215 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:02.867298 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:02.867723 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:02.868360 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:02.869310 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:02.869641 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:02.869751 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:02.872824 (+ 3073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.130:34417: BlockingRecv error: recv got EOF from 127.7.67.130:34417 (error 108)
Metrics: {"server-negotiator.queue_time_us":164}
Aug 11 05:14:03 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:03 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:03.064921 22145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:03.058545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.131:60255)
0811 05:14:03.058876 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:03.058898 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:03.059040 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:03.059401 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:03.059405 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:03.059643 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:03.059762 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:03.059769 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:03.060713 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:03.060718 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:03.061338 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:03.061347 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:03.061876 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:03.061880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:03.061979 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:03.062589 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:03.062602 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:03.064730 (+ 2128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":89,"threads_started":1}
W20260811 05:14:03.064981 22142 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:03.058650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:60255 (local address 127.7.67.190:37883)
0811 05:14:03.058826 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:03.058831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:03.058977 (+ 146us) server_negotiation.cc:407] Connection header received
0811 05:14:03.059203 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:03.059207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:03.059259 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:03.059369 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:03.059898 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:03.060504 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:03.061484 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:03.061739 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:03.061860 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:03.064862 (+ 3002us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.131:60255: BlockingRecv error: recv got EOF from 127.7.67.131:60255 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 05:14:03 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:03 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:03.654155 22146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:03.647954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.129:33807)
0811 05:14:03.648302 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:03.648317 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:03.648408 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:03.648714 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:03.648717 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:03.648912 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:03.649006 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:03.649011 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:03.650190 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:03.650196 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:03.650960 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:03.650971 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:03.651448 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:03.651451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:03.651513 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:03.652077 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:03.652096 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:03.653981 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":119,"threads_started":1}
W20260811 05:14:03.654241 22147 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:03.648027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:33807 (local address 127.7.67.190:37883)
0811 05:14:03.648452 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:03.648456 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:03.648469 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:14:03.648505 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:03.648507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:03.648601 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:03.648681 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:03.649200 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:03.650014 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:03.651077 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:03.651335 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:03.651426 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:03.654105 (+ 2679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.129:33807: BlockingRecv error: recv got EOF from 127.7.67.129:33807 (error 108)
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":132,"threads_started":1}
Aug 11 05:14:03 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:03 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:03.880069 22148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:03.873807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.130:52221)
0811 05:14:03.874205 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:03.874224 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:03.874366 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:03.874681 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:03.874684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:03.874926 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:03.875031 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:03.875037 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:03.875888 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:03.875894 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:03.876580 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:03.876590 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:03.877104 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:03.877108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:03.877211 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:03.877816 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:03.877829 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:03.879868 (+ 2039us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":139,"threads_started":1}
W20260811 05:14:03.880116 22147 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:03.874025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:52221 (local address 127.7.67.190:37883)
0811 05:14:03.874201 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:03.874207 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:03.874313 (+ 106us) server_negotiation.cc:407] Connection header received
0811 05:14:03.874487 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:03.874492 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:03.874551 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:03.874653 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:03.875163 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:03.875763 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:03.876722 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:03.876974 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:03.877097 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:03.880002 (+ 2905us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.130:52221: BlockingRecv error: recv got EOF from 127.7.67.130:52221 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 05:14:04 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:04 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:04.072039 22149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:04.065903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.131:43775)
0811 05:14:04.066186 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:04.066200 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:04.066331 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:04.066568 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:04.066570 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:04.066765 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:04.066858 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:04.066864 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:04.067644 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:04.067648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:04.068242 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:04.068251 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:04.068783 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:04.068786 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:04.068849 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:04.069485 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:04.069501 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:04.071863 (+ 2362us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":102,"threads_started":1}
W20260811 05:14:04.072095 22147 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:04.066024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:43775 (local address 127.7.67.190:37883)
0811 05:14:04.066171 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:04.066175 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:04.066342 (+ 167us) server_negotiation.cc:407] Connection header received
0811 05:14:04.066396 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:04.066399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:04.066449 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:04.066527 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:04.066977 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:04.067527 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:04.068367 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:04.068681 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:04.068767 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:04.071986 (+ 3219us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.131:43775: BlockingRecv error: recv got EOF from 127.7.67.131:43775 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 05:14:04 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:04 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:04.665649 22151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:04.655008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.129:37583)
0811 05:14:04.655313 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:04.655327 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:04.655420 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:04.655701 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:04.655705 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:04.655931 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:04.656027 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:04.656032 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:04.656925 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:04.656930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:04.657488 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:04.657496 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:04.657971 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:04.657976 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:04.658075 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:04.662941 (+ 4866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:04.662955 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:04.665458 (+ 2503us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":101,"threads_started":1}
W20260811 05:14:04.665755 22154 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:04.655054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:37583 (local address 127.7.67.190:37883)
0811 05:14:04.655356 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:04.655362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:04.655383 (+ 21us) server_negotiation.cc:407] Connection header received
0811 05:14:04.655522 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:04.655525 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:04.655573 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:04.655678 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:04.656160 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:04.656814 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:04.657611 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:04.657848 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:04.657957 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:04.665604 (+ 7647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.129:37583: BlockingRecv error: recv got EOF from 127.7.67.129:37583 (error 108)
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":96,"threads_started":1}
Aug 11 05:14:04 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:04 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:04.886898 22155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:04.880992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.130:39555)
0811 05:14:04.881300 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:04.881314 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:04.881420 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:04.881674 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:04.881676 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:04.881888 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:04.881986 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:04.881991 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:04.882770 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:04.882773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:04.883367 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:04.883376 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:04.883905 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:04.883908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:04.883980 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:04.884737 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:04.884753 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:04.886700 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":117,"threads_started":1}
W20260811 05:14:04.887009 22154 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:04.881086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:39555 (local address 127.7.67.190:37883)
0811 05:14:04.881237 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:04.881240 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:04.881389 (+ 149us) server_negotiation.cc:407] Connection header received
0811 05:14:04.881501 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:04.881503 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:04.881550 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:04.881626 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:04.882101 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:04.882639 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:04.883497 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:04.883736 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:04.883845 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:04.886882 (+ 3037us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.130:39555: BlockingRecv error: recv got EOF from 127.7.67.130:39555 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 05:14:05 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:05 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:05.078828 22156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:05.072994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.131:35935)
0811 05:14:05.073268 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:05.073287 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:05.073403 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:05.073721 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:05.073724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:05.074017 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:05.074174 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:05.074182 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:05.075131 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:05.075134 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:05.075696 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:05.075704 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:05.076164 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:05.076167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:05.076228 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:05.076863 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:05.076878 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:05.078704 (+ 1826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":92,"threads_started":1}
W20260811 05:14:05.079074 22154 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:05.073071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:35935 (local address 127.7.67.190:37883)
0811 05:14:05.073221 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:05.073226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:05.073384 (+ 158us) server_negotiation.cc:407] Connection header received
0811 05:14:05.073517 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:05.073520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:05.073584 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:05.073670 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:05.074336 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:05.075023 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:05.075832 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:05.076062 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:05.076146 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:05.078961 (+ 2815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.131:35935: BlockingRecv error: recv got EOF from 127.7.67.131:35935 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 05:14:05 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:05 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:05.672394 22158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:05.666566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.129:43027)
0811 05:14:05.666922 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:05.666936 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:05.667037 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:05.667268 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:05.667270 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:05.667468 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:05.667566 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:05.667570 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:05.668455 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:05.668458 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:05.669071 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:05.669081 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:05.669606 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:05.669611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:05.669679 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:05.670243 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:05.670255 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:05.672256 (+ 2001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":75,"threads_started":1}
W20260811 05:14:05.672577 22160 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:05.666612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:43027 (local address 127.7.67.190:37883)
0811 05:14:05.666923 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:05.666926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:05.667010 (+ 84us) server_negotiation.cc:407] Connection header received
0811 05:14:05.667103 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:05.667105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:05.667149 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:05.667227 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:05.667719 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:05.668350 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:05.669220 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:05.669473 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:05.669583 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:05.672415 (+ 2832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.129:43027: BlockingRecv error: recv got EOF from 127.7.67.129:43027 (error 108)
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":109,"threads_started":1}
Aug 11 05:14:05 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:05 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:05.894295 22161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:05.887862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.130:44357)
0811 05:14:05.888248 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:05.888263 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:05.888430 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:05.888835 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:05.888837 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:05.889040 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:05.889143 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:05.889149 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:05.889953 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:05.889956 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:05.890578 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:05.890587 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:05.891032 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:05.891035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:05.891104 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:05.891653 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:05.891667 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:05.894049 (+ 2382us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":123,"threads_started":1}
W20260811 05:14:05.894389 22160 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:05.887954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:44357 (local address 127.7.67.190:37883)
0811 05:14:05.888113 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:05.888117 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:05.888388 (+ 271us) server_negotiation.cc:407] Connection header received
0811 05:14:05.888641 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:05.888644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:05.888699 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:05.888776 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:05.889303 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:05.889835 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:05.890699 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:05.890921 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:05.891005 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:05.894239 (+ 3234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.130:44357: BlockingRecv error: recv got EOF from 127.7.67.130:44357 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 05:14:06 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:06 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:06.085624 22162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:06.079624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.131:36687)
0811 05:14:06.079964 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:06.079979 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:06.080084 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:06.080365 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:06.080368 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:06.080587 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:06.080681 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:06.080686 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:06.081566 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:06.081570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:06.082147 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:06.082155 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:06.082659 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:06.082661 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:06.082750 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:06.083380 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:06.083392 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:06.085455 (+ 2063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":127,"threads_started":1}
W20260811 05:14:06.085727 22160 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:06.079729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:36687 (local address 127.7.67.190:37883)
0811 05:14:06.079925 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:06.079931 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:06.080087 (+ 156us) server_negotiation.cc:407] Connection header received
0811 05:14:06.080167 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:06.080169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:06.080221 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:06.080299 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:06.080814 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:06.081436 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:06.082294 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:06.082549 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:06.082646 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:06.085593 (+ 2947us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.131:36687: BlockingRecv error: recv got EOF from 127.7.67.131:36687 (error 108)
Metrics: {"server-negotiator.queue_time_us":96}
Aug 11 05:14:06 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:06 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:06.678685 22163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:06.673207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.129:36871)
0811 05:14:06.673506 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:06.673520 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:06.673608 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:06.673871 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:06.673874 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:06.674061 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:06.674161 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:06.674166 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:06.674932 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:06.674935 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:06.675495 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:06.675504 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:06.676051 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:06.676057 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:06.676132 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:06.676808 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:06.676820 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:06.678561 (+ 1741us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":89,"threads_started":1}
W20260811 05:14:06.678862 22164 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:06.673348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:36871 (local address 127.7.67.190:37883)
0811 05:14:06.673596 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:06.673599 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:06.673613 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:14:06.673690 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:06.673693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:06.673749 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:06.673830 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:06.674273 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:06.674825 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:06.675628 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:06.675929 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:06.676014 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:06.678699 (+ 2685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.129:36871: BlockingRecv error: recv got EOF from 127.7.67.129:36871 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":95,"threads_started":1}
Aug 11 05:14:06 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:06 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:06.901284 22165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:06.895210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.130:41095)
0811 05:14:06.895593 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:06.895612 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:06.895755 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:06.896068 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:06.896073 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:06.896354 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:06.896501 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:06.896509 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:06.897503 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:06.897506 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:06.898073 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:06.898081 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:06.898537 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:06.898539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:06.898602 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:06.899229 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:06.899242 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:06.901134 (+ 1892us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":139,"threads_started":1}
W20260811 05:14:06.901415 22164 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:06.895368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:41095 (local address 127.7.67.190:37883)
0811 05:14:06.895545 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:06.895548 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:06.895718 (+ 170us) server_negotiation.cc:407] Connection header received
0811 05:14:06.895864 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:06.895869 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:06.895934 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:06.896000 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:06.896676 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:06.897372 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:06.898187 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:06.898435 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:06.898516 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:06.901294 (+ 2778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.130:41095: BlockingRecv error: recv got EOF from 127.7.67.130:41095 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 05:14:07 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:07 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:07.092988 22166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:07.086619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.131:40925)
0811 05:14:07.086930 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:07.086945 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:07.087051 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:07.087340 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:07.087343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:07.087605 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:07.087742 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:07.087749 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:07.088695 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:07.088701 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:07.089427 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:07.089442 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:07.089957 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:07.089960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:07.090029 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:07.090662 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:07.090681 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:07.092797 (+ 2116us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":120,"threads_started":1}
W20260811 05:14:07.093031 22164 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:07.086672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:40925 (local address 127.7.67.190:37883)
0811 05:14:07.086848 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:07.086854 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:07.087020 (+ 166us) server_negotiation.cc:407] Connection header received
0811 05:14:07.087135 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:07.087137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:07.087188 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:07.087296 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:07.087861 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:07.088480 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:07.089611 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:07.089844 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:07.089937 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:07.092929 (+ 2992us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.131:40925: BlockingRecv error: recv got EOF from 127.7.67.131:40925 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 05:14:07 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:07 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:07.685098 22170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:07.679500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.129:42417)
0811 05:14:07.679810 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:07.679824 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:07.679927 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:07.680214 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:07.680216 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:07.680408 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:07.680507 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:07.680512 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:07.681272 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:07.681274 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:07.681827 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:07.681835 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:07.682262 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:07.682264 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:07.682322 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:07.682844 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:07.682856 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:07.684926 (+ 2070us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":76,"threads_started":1}
W20260811 05:14:07.685173 22173 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:07.679563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:42417 (local address 127.7.67.190:37883)
0811 05:14:07.679864 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:07.679867 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:07.679880 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:14:07.680017 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:07.680019 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:07.680066 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:07.680142 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:07.680643 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:07.681175 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:07.681939 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:07.682162 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:07.682242 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:07.685047 (+ 2805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.129:42417: BlockingRecv error: recv got EOF from 127.7.67.129:42417 (error 108)
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":125,"threads_started":1}
Aug 11 05:14:07 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:07 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:07.907991 22174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:07.902123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.130:44471)
0811 05:14:07.902465 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:07.902479 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:07.902611 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:07.902901 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:07.902903 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:07.903121 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:07.903217 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:07.903253 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:07.904021 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:07.904025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:07.904668 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:07.904678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:07.905144 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:07.905148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:07.905244 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:07.905792 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:07.905804 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:07.907863 (+ 2059us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":105,"threads_started":1}
W20260811 05:14:07.908147 22173 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:07.902228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:44471 (local address 127.7.67.190:37883)
0811 05:14:07.902378 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:07.902383 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:07.902557 (+ 174us) server_negotiation.cc:407] Connection header received
0811 05:14:07.902717 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:07.902730 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:07.902783 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:07.902875 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:07.903372 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:07.903907 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:07.904793 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:07.905020 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:07.905129 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:07.908022 (+ 2893us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.130:44471: BlockingRecv error: recv got EOF from 127.7.67.130:44471 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 05:14:08 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:08 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:08.100885 22177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:08.093911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.131:41903)
0811 05:14:08.094263 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:08.094283 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:08.094418 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:08.094758 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:08.094763 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:08.095024 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:08.095174 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:08.095182 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:08.096257 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:08.096261 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:08.097122 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:08.097135 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:08.097773 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:08.097776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:08.097841 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:08.098475 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:08.098488 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:08.100743 (+ 2255us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":105,"threads_started":1}
W20260811 05:14:08.101012 22173 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:08.093994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:41903 (local address 127.7.67.190:37883)
0811 05:14:08.094181 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:08.094186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:08.094374 (+ 188us) server_negotiation.cc:407] Connection header received
0811 05:14:08.094535 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:08.094540 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:08.094596 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:08.094698 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:08.095319 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:08.096135 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:08.097278 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:08.097653 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:08.097772 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:08.100873 (+ 3101us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.131:41903: BlockingRecv error: recv got EOF from 127.7.67.131:41903 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 11 05:14:08 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:08 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:08.692001 22178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:08.686054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.129:56309)
0811 05:14:08.686369 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:08.686383 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:08.686478 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:08.686741 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:08.686743 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:08.686930 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:08.687029 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:08.687035 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:08.687912 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:08.687915 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:08.688516 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:08.688530 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:08.689174 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:08.689178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:08.689277 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:08.689907 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:08.689919 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:08.691873 (+ 1954us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":97,"threads_started":1}
W20260811 05:14:08.692273 22179 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:08.686134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:56309 (local address 127.7.67.190:37883)
0811 05:14:08.686371 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:08.686374 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:08.686450 (+ 76us) server_negotiation.cc:407] Connection header received
0811 05:14:08.686566 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:08.686568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:08.686613 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:08.686694 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:08.687128 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:08.687780 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:08.688669 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:08.689041 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:08.689169 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:08.692014 (+ 2845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.129:56309: BlockingRecv error: recv got EOF from 127.7.67.129:56309 (error 108)
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":93,"threads_started":1}
Aug 11 05:14:08 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:08 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:08.915138 22181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:08.908955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.130:50395)
0811 05:14:08.909340 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:08.909356 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:08.909508 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:08.909877 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:08.909880 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:08.910137 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:08.910289 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:08.910299 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:08.911324 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:08.911329 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:08.911903 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:08.911911 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:08.912396 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:08.912400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:08.912489 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:08.913047 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:08.913058 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:08.914956 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":94,"threads_started":1}
W20260811 05:14:08.915199 22179 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:08.909095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:50395 (local address 127.7.67.190:37883)
0811 05:14:08.909287 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:08.909292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:08.909486 (+ 194us) server_negotiation.cc:407] Connection header received
0811 05:14:08.909657 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:08.909661 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:08.909736 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:08.909851 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:08.910424 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:08.911200 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:08.912027 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:08.912271 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:08.912381 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:08.915083 (+ 2702us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.130:50395: BlockingRecv error: recv got EOF from 127.7.67.130:50395 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 11 05:14:09 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:09 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:09.107752 22184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:09.101669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.131:43075)
0811 05:14:09.102013 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:09.102029 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:09.102163 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:09.102478 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:09.102481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:09.102700 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:09.102804 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:09.102810 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:09.103588 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:09.103594 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:09.104161 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:09.104169 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:09.104718 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:09.104722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:09.104819 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:09.105368 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:09.105382 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:09.107553 (+ 2171us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":121,"threads_started":1}
W20260811 05:14:09.107795 22179 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:09.101853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:43075 (local address 127.7.67.190:37883)
0811 05:14:09.102013 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:09.102019 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:09.102114 (+ 95us) server_negotiation.cc:407] Connection header received
0811 05:14:09.102293 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:09.102296 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:09.102355 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:09.102447 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:09.102928 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:09.103473 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:09.104294 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:09.104589 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:09.104711 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:09.107683 (+ 2972us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.131:43075: BlockingRecv error: recv got EOF from 127.7.67.131:43075 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 11 05:14:09 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:09 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:09.702677 22186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:09.692804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.129:46203)
0811 05:14:09.693123 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:09.693149 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:09.693254 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:09.693508 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:09.693510 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:09.693712 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:09.693832 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:09.693838 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:09.694652 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:09.694656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:09.695394 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:09.695406 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:09.695861 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:09.695864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:09.695929 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:09.700514 (+ 4585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:09.700532 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:09.702559 (+ 2027us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":76,"threads_started":1}
W20260811 05:14:09.702998 22187 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:09.692956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:46203 (local address 127.7.67.190:37883)
0811 05:14:09.693237 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:09.693242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:09.693259 (+ 17us) server_negotiation.cc:407] Connection header received
0811 05:14:09.693326 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:09.693328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:09.693371 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:09.693459 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:09.693976 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:09.694537 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:09.695515 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:09.695751 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:09.695832 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:09.702717 (+ 6885us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.129:46203: BlockingRecv error: recv got EOF from 127.7.67.129:46203 (error 108)
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":84,"threads_started":1}
Aug 11 05:14:09 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:09 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:09.922163 22189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:09.916038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.130:33759)
0811 05:14:09.916386 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:09.916399 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:09.916500 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:09.916795 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:09.916798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:09.916989 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:09.917090 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:09.917095 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:09.917934 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:09.917937 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:09.918493 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:09.918501 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:09.919013 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:09.919016 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:09.919076 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:09.919753 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:09.919766 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:09.922003 (+ 2237us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":127,"threads_started":1}
W20260811 05:14:09.922343 22187 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:09.916137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:33759 (local address 127.7.67.190:37883)
0811 05:14:09.916290 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:09.916294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:09.916477 (+ 183us) server_negotiation.cc:407] Connection header received
0811 05:14:09.916602 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:09.916604 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:09.916652 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:09.916736 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:09.917242 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:09.917813 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:09.918630 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:09.918886 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:09.918997 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:09.922156 (+ 3159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.130:33759: BlockingRecv error: recv got EOF from 127.7.67.130:33759 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 05:14:10 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:10 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:10.114568 22190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:10.108625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.131:44219)
0811 05:14:10.108965 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:10.108987 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:10.109120 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:10.109448 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:10.109451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:10.109657 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:10.109748 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:10.109754 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:10.110518 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:10.110522 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:10.111085 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:10.111093 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:10.111566 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:10.111569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:10.111655 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:10.112148 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:10.112159 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:10.114379 (+ 2220us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":70,"threads_started":1}
W20260811 05:14:10.114635 22187 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:10.108687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:44219 (local address 127.7.67.190:37883)
0811 05:14:10.108833 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:10.108839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:10.109064 (+ 225us) server_negotiation.cc:407] Connection header received
0811 05:14:10.109245 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:10.109250 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:10.109309 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:10.109392 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:10.109868 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:10.110404 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:10.111212 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:10.111442 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:10.111550 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:10.114516 (+ 2966us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.131:44219: BlockingRecv error: recv got EOF from 127.7.67.131:44219 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 05:14:10 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:10 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:10.711405 22191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:10.703761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.129:58813)
0811 05:14:10.705717 (+ 1956us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:10.705735 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:10.706075 (+ 340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:10.706368 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:10.706370 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:10.706551 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:10.706649 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:10.706654 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:10.707434 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:10.707437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:10.708081 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:10.708091 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:10.708593 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:10.708596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:10.708656 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:10.709201 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:10.709213 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:10.711228 (+ 2015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1856,"thread_start_us":1739,"threads_started":1}
W20260811 05:14:10.711480 22192 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:10.703845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:58813 (local address 127.7.67.190:37883)
0811 05:14:10.705719 (+ 1874us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:10.705725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:10.705842 (+ 117us) server_negotiation.cc:407] Connection header received
0811 05:14:10.706183 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:10.706186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:10.706242 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:10.706329 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:10.706759 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:10.707322 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:10.708206 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:10.708435 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:10.708598 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:10.711353 (+ 2755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.129:58813: BlockingRecv error: recv got EOF from 127.7.67.129:58813 (error 108)
Metrics: {"server-negotiator.queue_time_us":1796,"thread_start_us":1686,"threads_started":1}
Aug 11 05:14:10 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:10 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:10.929045 22194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:10.923177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.130:52197)
0811 05:14:10.923505 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:10.923528 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:10.923663 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:10.923948 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:10.923952 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:10.924170 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:10.924289 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:10.924295 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:10.925158 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:10.925163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:10.925729 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:10.925737 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:10.926228 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:10.926232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:10.926327 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:10.926867 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:10.926879 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:10.928858 (+ 1979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":98,"threads_started":1}
W20260811 05:14:10.929100 22192 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:10.923291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:52197 (local address 127.7.67.190:37883)
0811 05:14:10.923476 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:10.923481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:10.923603 (+ 122us) server_negotiation.cc:407] Connection header received
0811 05:14:10.923769 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:10.923772 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:10.923821 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:10.923922 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:10.924412 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:10.925043 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:10.925854 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:10.926098 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:10.926207 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:10.928988 (+ 2781us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.130:52197: BlockingRecv error: recv got EOF from 127.7.67.130:52197 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 05:14:11 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:11 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:11.121644 22195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:11.115464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.131:52855)
0811 05:14:11.115806 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:11.115820 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:11.115922 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:11.116222 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:11.116225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:11.116466 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:11.116638 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:11.116645 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:11.117688 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:11.117692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:11.118244 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:11.118252 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:11.118675 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:11.118678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:11.118740 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:11.119367 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:11.119380 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:11.121471 (+ 2091us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":124,"threads_started":1}
W20260811 05:14:11.121696 22192 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:11.115534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:52855 (local address 127.7.67.190:37883)
0811 05:14:11.115690 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:11.115696 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:11.115896 (+ 200us) server_negotiation.cc:407] Connection header received
0811 05:14:11.116004 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:11.116006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:11.116055 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:11.116123 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:11.116736 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:11.117534 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:11.118351 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:11.118565 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:11.118646 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:11.121589 (+ 2943us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.131:52855: BlockingRecv error: recv got EOF from 127.7.67.131:52855 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 05:14:11 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:11 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:11.717517 22196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:11.712115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.129:38855)
0811 05:14:11.712432 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:11.712446 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:11.712559 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:11.712809 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:11.712811 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:11.713004 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:11.713098 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:11.713103 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:11.713876 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:11.713879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:11.714433 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:11.714441 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:11.714877 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:11.714879 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:11.714939 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:11.715514 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:11.715526 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:11.717404 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":80,"threads_started":1}
W20260811 05:14:11.717727 22197 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:11.712199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:38855 (local address 127.7.67.190:37883)
0811 05:14:11.712473 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:11.712476 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:11.712490 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:14:11.712645 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:11.712647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:11.712697 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:11.712772 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:11.713202 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:11.713771 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:11.714545 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:11.714779 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:11.714856 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:11.717539 (+ 2683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.129:38855: BlockingRecv error: recv got EOF from 127.7.67.129:38855 (error 108)
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":92,"threads_started":1}
Aug 11 05:14:11 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:11 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:11.935576 22198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:11.929922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.130:45689)
0811 05:14:11.930346 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:11.930358 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:11.930455 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:11.930701 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:11.930703 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:11.930890 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:11.930986 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:11.930991 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:11.931760 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:11.931763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:11.932342 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:11.932350 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:11.932866 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:11.932869 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:11.932939 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:11.933498 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:11.933511 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:11.935462 (+ 1951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":113,"threads_started":1}
W20260811 05:14:11.935732 22197 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:11.930112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:45689 (local address 127.7.67.190:37883)
0811 05:14:11.930254 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:11.930258 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:11.930436 (+ 178us) server_negotiation.cc:407] Connection header received
0811 05:14:11.930537 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:11.930539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:11.930586 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:11.930660 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:11.931100 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:11.931646 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:11.932478 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:11.932739 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:11.932834 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:11.935610 (+ 2776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.130:45689: BlockingRecv error: recv got EOF from 127.7.67.130:45689 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 11 05:14:12 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:12 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:12.128249 22199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:12.122445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.131:54763)
0811 05:14:12.122740 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:12.122754 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:12.122854 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:12.123084 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:12.123086 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:12.123280 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:12.123377 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:12.123382 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:12.124094 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:12.124096 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:12.124740 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:12.124752 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:12.125257 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:12.125260 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:12.125328 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:12.125987 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:12.126006 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:12.128113 (+ 2107us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":115,"threads_started":1}
W20260811 05:14:12.128389 22197 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:12.122535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:54763 (local address 127.7.67.190:37883)
0811 05:14:12.122676 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:12.122680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:12.122853 (+ 173us) server_negotiation.cc:407] Connection header received
0811 05:14:12.122918 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:12.122921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:12.122971 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:12.123045 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:12.123485 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:12.124000 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:12.124871 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:12.125094 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:12.125198 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:12.128240 (+ 3042us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.131:54763: BlockingRecv error: recv got EOF from 127.7.67.131:54763 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 05:14:12 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:12 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:12.724104 22200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:12.718375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.129:44197)
0811 05:14:12.718694 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:12.718709 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:12.718805 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:12.719061 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:12.719063 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:12.719261 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:12.719358 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:12.719364 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:12.720137 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:12.720140 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:12.720754 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:12.720764 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:12.721308 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:12.721311 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:12.721377 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:12.721922 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:12.721933 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:12.723929 (+ 1996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":99,"threads_started":1}
W20260811 05:14:12.724192 22202 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:12.718445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:44197 (local address 127.7.67.190:37883)
0811 05:14:12.718764 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:12.718768 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:12.718781 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:14:12.718897 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:12.718899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:12.718944 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:12.719020 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:12.719465 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:12.720009 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:12.720892 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:12.721159 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:12.721287 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:12.724054 (+ 2767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.129:44197: BlockingRecv error: recv got EOF from 127.7.67.129:44197 (error 108)
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":63,"threads_started":1}
Aug 11 05:14:12 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:12 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:12.942085 22205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:12.936578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.130:44095)
0811 05:14:12.936889 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:12.936903 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:12.937003 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:12.937232 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:12.937234 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:12.937425 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:12.937522 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:12.937527 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:12.938254 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:12.938256 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:12.938801 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:12.938807 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:12.939264 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:12.939266 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:12.939327 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:12.939820 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:12.939842 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:12.941911 (+ 2069us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":103,"threads_started":1}
W20260811 05:14:12.942162 22202 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:12.936654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:44095 (local address 127.7.67.190:37883)
0811 05:14:12.936810 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:12.936817 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:12.936979 (+ 162us) server_negotiation.cc:407] Connection header received
0811 05:14:12.937069 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:12.937071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:12.937116 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:12.937197 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:12.937625 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:12.938155 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:12.938929 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:12.939153 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:12.939246 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:12.942044 (+ 2798us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.130:44095: BlockingRecv error: recv got EOF from 127.7.67.130:44095 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 05:14:13 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:13 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.131@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:13.135191 22206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:13.129019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.131:60685)
0811 05:14:13.129372 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:13.129385 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:13.129525 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:13.129808 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:13.129812 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:13.130042 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:13.130140 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:13.130147 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:13.130918 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:13.130922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:13.131501 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:13.131509 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:13.132142 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:13.132145 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:13.132265 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:13.132923 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:13.132938 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:13.135058 (+ 2120us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":93,"threads_started":1}
W20260811 05:14:13.135479 22202 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:13.129152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:60685 (local address 127.7.67.190:37883)
0811 05:14:13.129323 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:13.129328 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:13.129461 (+ 133us) server_negotiation.cc:407] Connection header received
0811 05:14:13.129628 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:13.129631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:13.129692 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:13.129785 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:13.130259 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:13.130801 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:13.131635 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:13.131983 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:13.132107 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:13.135243 (+ 3136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.131:60685: BlockingRecv error: recv got EOF from 127.7.67.131:60685 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 05:14:13 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:13 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.129@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:13.730955 22208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:13.725281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.129:42043)
0811 05:14:13.725617 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:13.725631 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:13.725771 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:13.726098 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:13.726100 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:13.726298 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:13.726407 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:13.726413 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:13.727166 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:13.727169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:13.727761 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:13.727770 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:13.728241 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:13.728244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:13.728308 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:13.728861 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:13.728876 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:13.730826 (+ 1950us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":87,"threads_started":1}
W20260811 05:14:13.731164 22207 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:13.725293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:42043 (local address 127.7.67.190:37883)
0811 05:14:13.725562 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:13.725567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:13.725712 (+ 145us) server_negotiation.cc:407] Connection header received
0811 05:14:13.725881 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:13.725884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:13.725943 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:13.726055 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:13.726524 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:13.727060 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:13.727883 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:13.728115 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:13.728194 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:13.730980 (+ 2786us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.129:42043: BlockingRecv error: recv got EOF from 127.7.67.129:42043 (error 108)
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":84,"threads_started":1}
Aug 11 05:14:13 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 05:14:13 dist-test-slave-04bb krb5kdc[20589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.7.67.130@KRBTEST.COM for oryx/127.7.67.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 05:14:13.949182 22209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:13.943139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37883 (local address 127.7.67.130:42499)
0811 05:14:13.943406 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:13.943428 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:13.943525 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:13.943772 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:13.943774 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:13.943959 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:13.944055 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:13.944060 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:13.944891 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:13.944894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:13.945452 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:13.945461 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:13.945934 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:13.945936 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:13.945998 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:13.946669 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:13.946687 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:13.949058 (+ 2371us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.7.67.190:37883: Server oryx/127.7.67.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":82,"threads_started":1}
W20260811 05:14:13.949347 22207 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:13.943202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:42499 (local address 127.7.67.190:37883)
0811 05:14:13.943351 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:13.943355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:13.943501 (+ 146us) server_negotiation.cc:407] Connection header received
0811 05:14:13.943610 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:13.943613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:13.943660 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:13.943731 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:13.944204 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:13.944780 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:13.945577 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:13.945829 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:13.945919 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:13.949190 (+ 3271us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.7.67.130:42499: BlockingRecv error: recv got EOF from 127.7.67.130:42499 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:14:14.027267 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 21486
I20260811 05:14:14.044935 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 21719
I20260811 05:14:14.066421 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 21917
I20260811 05:14:14.080919 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 21385
2026-08-11T05:14:14Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18368 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T05:14:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:14:14Z Disabled control of system clock
I20260811 05:14:14.120221 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:35873
--webserver_interface=127.7.67.190
--webserver_port=0
--builtin_ntp_servers=127.7.67.148:39433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:35873
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 05:14:14.233560 22213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:14:14.233873 22213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:14:14.233940 22213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:14:14.237943 22213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:14:14.238044 22213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:14:14.238071 22213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:14:14.238113 22213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:14:14.238144 22213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:14:14.242882 22213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:39433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:35873
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:35873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:14:14.244007 22213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:14:14.245498 22213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:14:14.254655 22218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:14:14.254747 22213 server_base.cc:1034] running on GCE node
W20260811 05:14:14.254767 22219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:14.254990 22223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:14:14.255555 22213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:14:14.256728 22213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:14:14.257896 22213 hybrid_clock.cc:648] HybridClock initialized: now 1786425254257872 us; error 43 us; skew 500 ppm
I20260811 05:14:14.259833 22213 webserver.cc:460] Webserver started at http://127.7.67.190:39053/ using document root <none> and password file <none>
I20260811 05:14:14.260454 22213 fs_manager.cc:359] Metadata directory not provided
I20260811 05:14:14.260571 22213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:14:14.260833 22213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:14:14.263031 22213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "1a10058bb05041caa7f501a06846bd5f"
format_stamp: "Formatted at 2026-08-11 05:14:14 on dist-test-slave-04bb"
server_key: "57d3013fade6c76f51fad3f06b3857eb"
server_key_iv: "382a40b4fb875d31cdecf208877bbf6e"
server_key_version: "encryptionkey@0"
I20260811 05:14:14.263638 22213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "1a10058bb05041caa7f501a06846bd5f"
format_stamp: "Formatted at 2026-08-11 05:14:14 on dist-test-slave-04bb"
server_key: "57d3013fade6c76f51fad3f06b3857eb"
server_key_iv: "382a40b4fb875d31cdecf208877bbf6e"
server_key_version: "encryptionkey@0"
I20260811 05:14:14.267406 22213 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260811 05:14:14.269985 22231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:14.271343 22213 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:14:14.271510 22213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "1a10058bb05041caa7f501a06846bd5f"
format_stamp: "Formatted at 2026-08-11 05:14:14 on dist-test-slave-04bb"
server_key: "57d3013fade6c76f51fad3f06b3857eb"
server_key_iv: "382a40b4fb875d31cdecf208877bbf6e"
server_key_version: "encryptionkey@0"
I20260811 05:14:14.271647 22213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:14:14.293936 22213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:14:14.294703 22213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:14:14.294934 22213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:14:14.302815 22213 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:35873
I20260811 05:14:14.302824 22319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:35873 every 8 connection(s)
I20260811 05:14:14.303828 22213 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:14:14.306959 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 22213
I20260811 05:14:14.307076 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:14:14.307122 22320 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:14:14.307430 7438 external_mini_cluster.cc:1428] Setting key 7df92b1587cced457bd0f9da41127dc1
I20260811 05:14:14.313902 22320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a10058bb05041caa7f501a06846bd5f: Bootstrap starting.
I20260811 05:14:14.316376 22320 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a10058bb05041caa7f501a06846bd5f: Neither blocks nor log segments found. Creating new log.
I20260811 05:14:14.317167 22320 log.cc:824] T 00000000000000000000000000000000 P 1a10058bb05041caa7f501a06846bd5f: Log is configured to *not* fsync() on all Append() calls
I20260811 05:14:14.317448 22324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:14.308961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35552 (local address 127.7.67.190:35873)
0811 05:14:14.309708 (+ 747us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:14.309725 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:14.309763 (+ 38us) server_negotiation.cc:407] Connection header received
0811 05:14:14.310476 (+ 713us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:14.310495 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:14.310936 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:14.311278 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:14.311721 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:14.312735 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:14.313902 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:14.314428 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:14.314631 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:14.315560 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:14:14.315565 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:14:14.315577 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:14:14.316003 (+ 426us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:14:14.316060 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:14:14.316080 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:14:14.316195 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:14.316365 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 05:14:14.316686 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":147,"threads_started":1}
I20260811 05:14:14.319259 22320 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a10058bb05041caa7f501a06846bd5f: No bootstrap required, opened a new log
I20260811 05:14:14.322868 22320 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a10058bb05041caa7f501a06846bd5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a10058bb05041caa7f501a06846bd5f" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 35873 } }
I20260811 05:14:14.323105 22320 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1a10058bb05041caa7f501a06846bd5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:14:14.323190 22320 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a10058bb05041caa7f501a06846bd5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a10058bb05041caa7f501a06846bd5f, State: Initialized, Role: FOLLOWER
I20260811 05:14:14.323614 22320 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a10058bb05041caa7f501a06846bd5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a10058bb05041caa7f501a06846bd5f" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 35873 } }
I20260811 05:14:14.323764 22320 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1a10058bb05041caa7f501a06846bd5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:14:14.323856 22320 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a10058bb05041caa7f501a06846bd5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:14:14.323987 22320 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a10058bb05041caa7f501a06846bd5f [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:14:14.324904 22320 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a10058bb05041caa7f501a06846bd5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a10058bb05041caa7f501a06846bd5f" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 35873 } }
I20260811 05:14:14.325268 22320 leader_election.cc:302] T 00000000000000000000000000000000 P 1a10058bb05041caa7f501a06846bd5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1a10058bb05041caa7f501a06846bd5f; no voters:
I20260811 05:14:14.325681 22320 leader_election.cc:288] T 00000000000000000000000000000000 P 1a10058bb05041caa7f501a06846bd5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:14:14.325824 22331 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a10058bb05041caa7f501a06846bd5f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:14:14.326052 22331 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a10058bb05041caa7f501a06846bd5f [term 1 LEADER]: Becoming Leader. State: Replica: 1a10058bb05041caa7f501a06846bd5f, State: Running, Role: LEADER
I20260811 05:14:14.326398 22331 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a10058bb05041caa7f501a06846bd5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a10058bb05041caa7f501a06846bd5f" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 35873 } }
I20260811 05:14:14.326872 22320 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a10058bb05041caa7f501a06846bd5f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:14:14.328218 22333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a10058bb05041caa7f501a06846bd5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a10058bb05041caa7f501a06846bd5f. Latest consensus state: current_term: 1 leader_uuid: "1a10058bb05041caa7f501a06846bd5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a10058bb05041caa7f501a06846bd5f" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 35873 } } }
I20260811 05:14:14.328370 22333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a10058bb05041caa7f501a06846bd5f [sys.catalog]: This master's current role is: LEADER
I20260811 05:14:14.328661 22332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a10058bb05041caa7f501a06846bd5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1a10058bb05041caa7f501a06846bd5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1a10058bb05041caa7f501a06846bd5f" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 35873 } } }
I20260811 05:14:14.328768 22332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a10058bb05041caa7f501a06846bd5f [sys.catalog]: This master's current role is: LEADER
I20260811 05:14:14.328841 22344 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:14:14.331888 22344 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:14:14.337162 22344 catalog_manager.cc:1347] Generated new cluster ID: c0b94280b08d41349922c70797d4410d
I20260811 05:14:14.337229 22344 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:14:14.347522 22344 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:14:14.348436 22344 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:14:14.356635 22344 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a10058bb05041caa7f501a06846bd5f: Generated new TSK 0
I20260811 05:14:14.357299 22344 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:14:14.364241 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:35873
--builtin_ntp_servers=127.7.67.148:39433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 05:14:14.482540 22359 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:14:14.482803 22359 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:14:14.482842 22359 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:14:14.487540 22359 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:14:14.487600 22359 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:14:14.487654 22359 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:14:14.492961 22359 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:39433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:35873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22359
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:14:14.494462 22359 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:14:14.496220 22359 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:14:14.506407 22359 server_base.cc:1034] running on GCE node
W20260811 05:14:14.506654 22370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:14.506677 22369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:14.506677 22372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:14:14.507409 22359 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:14:14.508064 22359 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:14:14.509282 22359 hybrid_clock.cc:648] HybridClock initialized: now 1786425254509269 us; error 31 us; skew 500 ppm
I20260811 05:14:14.511272 22359 webserver.cc:460] Webserver started at http://127.7.67.129:38693/ using document root <none> and password file <none>
I20260811 05:14:14.511868 22359 fs_manager.cc:359] Metadata directory not provided
I20260811 05:14:14.511965 22359 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:14:14.512217 22359 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:14:14.514394 22359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "85ce52e22beb476a83acd25e4acff6df"
format_stamp: "Formatted at 2026-08-11 05:14:14 on dist-test-slave-04bb"
server_key: "7382295fcf2f9f810df718af2da79dfa"
server_key_iv: "381e53484319e07ad9943a281627ee39"
server_key_version: "encryptionkey@0"
I20260811 05:14:14.515019 22359 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "85ce52e22beb476a83acd25e4acff6df"
format_stamp: "Formatted at 2026-08-11 05:14:14 on dist-test-slave-04bb"
server_key: "7382295fcf2f9f810df718af2da79dfa"
server_key_iv: "381e53484319e07ad9943a281627ee39"
server_key_version: "encryptionkey@0"
I20260811 05:14:14.518852 22359 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 05:14:14.521624 22380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:14.522869 22359 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:14:14.523016 22359 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "85ce52e22beb476a83acd25e4acff6df"
format_stamp: "Formatted at 2026-08-11 05:14:14 on dist-test-slave-04bb"
server_key: "7382295fcf2f9f810df718af2da79dfa"
server_key_iv: "381e53484319e07ad9943a281627ee39"
server_key_version: "encryptionkey@0"
I20260811 05:14:14.523139 22359 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:14:14.540673 22359 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:14:14.541409 22359 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:14:14.541622 22359 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:14:14.542246 22359 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:14:14.543253 22359 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:14:14.543327 22359 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:14.543401 22359 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:14:14.543443 22359 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:14.552874 22359 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:39169
I20260811 05:14:14.552922 22562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:39169 every 8 connection(s)
I20260811 05:14:14.553840 22359 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:14:14.562119 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 22359
I20260811 05:14:14.562247 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:14:14.562506 7438 external_mini_cluster.cc:1428] Setting key 59a80375e505b5ab27dd3285078db7d0
I20260811 05:14:14.563544 22324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:14.555755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:46065 (local address 127.7.67.190:35873)
0811 05:14:14.555940 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:14.555945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:14.557127 (+ 1182us) server_negotiation.cc:407] Connection header received
0811 05:14:14.558506 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:14.558510 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:14.558558 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:14.558640 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:14.559562 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:14.560129 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:14.561398 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:14.561621 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:14.561721 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:14.563035 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:14:14.563038 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:14:14.563040 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:14:14.563054 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:14:14.563073 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:14:14.563081 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:14:14.563172 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:14.563375 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 05:14:14.563431 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 05:14:14.564641 22566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:14.556190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:35873 (local address 127.7.67.129:46065)
0811 05:14:14.556953 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:14.556998 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:14.558285 (+ 1287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:14.558818 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:14.558826 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:14.558906 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:14.559400 (+ 494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:14.559413 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:14.560269 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:14.560272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:14.561250 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:14.561261 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:14.561779 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:14.561782 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:14.561947 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:14.562655 (+ 708us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:14:14.562657 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:14.562715 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:14:14.562840 (+ 125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:14:14.563196 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:14:14.563205 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:14:14.563210 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:14:14.563359 (+ 149us) client_negotiation.cc:240] Negotiation successful
0811 05:14:14.563646 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":128,"threads_started":1}
I20260811 05:14:14.564766 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:35873
--builtin_ntp_servers=127.7.67.148:39433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 05:14:14.566021 22564 heartbeater.cc:347] Connected to a master server at 127.7.67.190:35873
I20260811 05:14:14.566286 22564 heartbeater.cc:464] Registering TS with master...
I20260811 05:14:14.566910 22564 heartbeater.cc:511] Master 127.7.67.190:35873 requested a full tablet report, sending...
I20260811 05:14:14.568735 22266 ts_manager.cc:194] Registered new tserver with Master: 85ce52e22beb476a83acd25e4acff6df (127.7.67.129:39169)
I20260811 05:14:14.570489 22266 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.67.129:46065
W20260811 05:14:14.679486 22567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:14:14.679723 22567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:14:14.679765 22567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:14:14.683578 22567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:14:14.683640 22567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:14:14.683689 22567 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:14:14.688222 22567 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:39433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:35873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:14:14.689278 22567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:14:14.690587 22567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:14:14.700253 22580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:14.700359 22576 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:14.700294 22577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:14:14.700500 22567 server_base.cc:1034] running on GCE node
I20260811 05:14:14.701016 22567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:14:14.701622 22567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:14:14.702831 22567 hybrid_clock.cc:648] HybridClock initialized: now 1786425254702797 us; error 47 us; skew 500 ppm
I20260811 05:14:14.710150 22567 webserver.cc:460] Webserver started at http://127.7.67.130:35361/ using document root <none> and password file <none>
I20260811 05:14:14.710816 22567 fs_manager.cc:359] Metadata directory not provided
I20260811 05:14:14.710912 22567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:14:14.711158 22567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:14:14.713409 22567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "0a155c0de42146099c899e6bb1c85e8e"
format_stamp: "Formatted at 2026-08-11 05:14:14 on dist-test-slave-04bb"
server_key: "cb87e974cd0e618d49fc6a3ebd71e093"
server_key_iv: "4cfae6757674f1cdcee89dd53d3263f0"
server_key_version: "encryptionkey@0"
I20260811 05:14:14.713984 22567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "0a155c0de42146099c899e6bb1c85e8e"
format_stamp: "Formatted at 2026-08-11 05:14:14 on dist-test-slave-04bb"
server_key: "cb87e974cd0e618d49fc6a3ebd71e093"
server_key_iv: "4cfae6757674f1cdcee89dd53d3263f0"
server_key_version: "encryptionkey@0"
I20260811 05:14:14.717739 22567 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:14:14.720288 22587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:14.721639 22567 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:14:14.721803 22567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "0a155c0de42146099c899e6bb1c85e8e"
format_stamp: "Formatted at 2026-08-11 05:14:14 on dist-test-slave-04bb"
server_key: "cb87e974cd0e618d49fc6a3ebd71e093"
server_key_iv: "4cfae6757674f1cdcee89dd53d3263f0"
server_key_version: "encryptionkey@0"
I20260811 05:14:14.721931 22567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:14:14.757658 22567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:14:14.758447 22567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:14:14.758661 22567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:14:14.759311 22567 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:14:14.760380 22567 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:14:14.760455 22567 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:14.760527 22567 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:14:14.760592 22567 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:14.769939 22567 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:43333
I20260811 05:14:14.769963 22769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:43333 every 8 connection(s)
I20260811 05:14:14.770859 22567 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:14:14.773595 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 22567
I20260811 05:14:14.773712 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:14:14.774063 7438 external_mini_cluster.cc:1428] Setting key e1adc35ee7244ba763d64014975bcab9
I20260811 05:14:14.776432 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:35873
--builtin_ntp_servers=127.7.67.148:39433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 05:14:14.781311 22324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:14.772798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:50307 (local address 127.7.67.190:35873)
0811 05:14:14.772970 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:14.772975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:14.774314 (+ 1339us) server_negotiation.cc:407] Connection header received
0811 05:14:14.776130 (+ 1816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:14.776132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:14.776177 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:14.776276 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:14.777559 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:14.778071 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:14.779274 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:14.779491 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:14.779608 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:14.780790 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:14:14.780794 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:14:14.780797 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:14:14.780816 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:14:14.780835 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:14:14.780843 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:14:14.780922 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:14.781090 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 05:14:14.781157 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 05:14:14.782160 22772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:14.773292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:35873 (local address 127.7.67.130:50307)
0811 05:14:14.774103 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:14.774165 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:14.775879 (+ 1714us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:14.776494 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:14.776519 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:14.776655 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:14.777395 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:14.777417 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:14.778202 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:14.778206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:14.779097 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:14.779107 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:14.779650 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:14.779654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:14.779828 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:14.780473 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:14:14.780474 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:14.780515 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:14:14.780631 (+ 116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:14:14.780926 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:14:14.780939 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:14:14.780946 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 05:14:14.781076 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 05:14:14.781352 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":116,"threads_started":1}
I20260811 05:14:14.783430 22770 heartbeater.cc:347] Connected to a master server at 127.7.67.190:35873
I20260811 05:14:14.783705 22770 heartbeater.cc:464] Registering TS with master...
I20260811 05:14:14.784237 22770 heartbeater.cc:511] Master 127.7.67.190:35873 requested a full tablet report, sending...
I20260811 05:14:14.785451 22266 ts_manager.cc:194] Registered new tserver with Master: 0a155c0de42146099c899e6bb1c85e8e (127.7.67.130:43333)
I20260811 05:14:14.786350 22266 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.67.130:50307
W20260811 05:14:14.894100 22776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:14:14.894361 22776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:14:14.894399 22776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:14:14.898609 22776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:14:14.898664 22776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:14:14.898716 22776 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:14:14.903355 22776 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:39433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:35873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:14:14.904331 22776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:14:14.905756 22776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:14:14.915632 22797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:14.915644 22788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:14.915647 22794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:14:14.915863 22776 server_base.cc:1034] running on GCE node
I20260811 05:14:14.916313 22776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:14:14.917042 22776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:14:14.918251 22776 hybrid_clock.cc:648] HybridClock initialized: now 1786425254918213 us; error 48 us; skew 500 ppm
I20260811 05:14:14.920138 22776 webserver.cc:460] Webserver started at http://127.7.67.131:44611/ using document root <none> and password file <none>
I20260811 05:14:14.920812 22776 fs_manager.cc:359] Metadata directory not provided
I20260811 05:14:14.920909 22776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:14:14.921152 22776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:14:14.923216 22776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "e54fd0c352284b11937058b6e7abece3"
format_stamp: "Formatted at 2026-08-11 05:14:14 on dist-test-slave-04bb"
server_key: "9381a42154e8c2a7212e9d3efade79e9"
server_key_iv: "5601ddcc8dc662ad905ad9002e3abbd5"
server_key_version: "encryptionkey@0"
I20260811 05:14:14.923799 22776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "e54fd0c352284b11937058b6e7abece3"
format_stamp: "Formatted at 2026-08-11 05:14:14 on dist-test-slave-04bb"
server_key: "9381a42154e8c2a7212e9d3efade79e9"
server_key_iv: "5601ddcc8dc662ad905ad9002e3abbd5"
server_key_version: "encryptionkey@0"
I20260811 05:14:14.927990 22776 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 05:14:14.930675 22812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:14.932054 22776 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:14:14.932222 22776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "e54fd0c352284b11937058b6e7abece3"
format_stamp: "Formatted at 2026-08-11 05:14:14 on dist-test-slave-04bb"
server_key: "9381a42154e8c2a7212e9d3efade79e9"
server_key_iv: "5601ddcc8dc662ad905ad9002e3abbd5"
server_key_version: "encryptionkey@0"
I20260811 05:14:14.932358 22776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:14:14.947149 22776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:14:14.947907 22776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:14:14.948128 22776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:14:14.948837 22776 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:14:14.949870 22776 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:14:14.949947 22776 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:14.950027 22776 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:14:14.950075 22776 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:14.959848 22776 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:40291
I20260811 05:14:14.959877 22986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:40291 every 8 connection(s)
I20260811 05:14:14.960888 22776 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:14:14.963937 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 22776
I20260811 05:14:14.964038 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:14:14.964360 7438 external_mini_cluster.cc:1428] Setting key b9ab8e0b7ec2e88d0b04b714d0f453c3
I20260811 05:14:14.970930 22324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:14.962726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:43771 (local address 127.7.67.190:35873)
0811 05:14:14.962882 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:14.962888 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:14.964198 (+ 1310us) server_negotiation.cc:407] Connection header received
0811 05:14:14.965893 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:14.965896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:14.965944 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:14.966051 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:14.967020 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:14.967600 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:14.968938 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:14.969181 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:14.969312 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:14.970393 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:14:14.970397 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:14:14.970400 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:14:14.970421 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:14:14.970440 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:14:14.970449 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:14:14.970536 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:14.970711 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 05:14:14.970777 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 05:14:14.971848 22990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:14.963190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:35873 (local address 127.7.67.131:43771)
0811 05:14:14.963985 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:14.964062 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:14.965577 (+ 1515us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:14.966244 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:14.966251 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:14.966343 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:14.966862 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:14.966876 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:14.967741 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:14.967747 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:14.968801 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:14.968811 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:14.969343 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:14.969349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:14.969534 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:14.970119 (+ 585us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:14:14.970121 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:14.970162 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:14:14.970230 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:14:14.970538 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:14:14.970551 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:14:14.970561 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 05:14:14.970708 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 05:14:14.971014 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":123,"threads_started":1}
I20260811 05:14:14.973033 22987 heartbeater.cc:347] Connected to a master server at 127.7.67.190:35873
I20260811 05:14:14.973306 22987 heartbeater.cc:464] Registering TS with master...
I20260811 05:14:14.973855 22987 heartbeater.cc:511] Master 127.7.67.190:35873 requested a full tablet report, sending...
I20260811 05:14:14.974961 22266 ts_manager.cc:194] Registered new tserver with Master: e54fd0c352284b11937058b6e7abece3 (127.7.67.131:40291)
I20260811 05:14:14.975648 22266 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.67.131:43771
I20260811 05:14:14.978319 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 05:14:14.981352 22324 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:14.980348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35562 (local address 127.7.67.190:35873)
0811 05:14:14.980510 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:14.980515 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:14.980700 (+ 185us) server_negotiation.cc:407] Connection header received
0811 05:14:14.980826 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:14.980830 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:14.980878 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:14.980963 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:14.981260 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35562: BlockingRecv error: recv got EOF from 127.0.0.1:35562 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 05:14:14.981696 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 22359
I20260811 05:14:15.001955 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 22567
I20260811 05:14:15.032428 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 22776
I20260811 05:14:15.042575 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 22213
2026-08-11T05:14:15Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (958 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T05:14:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:14:15Z Disabled control of system clock
I20260811 05:14:15.078573 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:40683
--webserver_interface=127.7.67.190
--webserver_port=0
--builtin_ntp_servers=127.7.67.148:44151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:40683
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 05:14:15.193382 23005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:14:15.193648 23005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:14:15.193687 23005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:14:15.197592 23005 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:14:15.197657 23005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:14:15.197674 23005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:14:15.197690 23005 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:14:15.197705 23005 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:14:15.202405 23005 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:44151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:40683
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:40683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:14:15.203485 23005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:14:15.205006 23005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:14:15.213841 23011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:14:15.213923 23005 server_base.cc:1034] running on GCE node
W20260811 05:14:15.213860 23013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:15.213847 23015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:14:15.214707 23005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:14:15.215745 23005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:14:15.216933 23005 hybrid_clock.cc:648] HybridClock initialized: now 1786425255216920 us; error 27 us; skew 500 ppm
I20260811 05:14:15.219061 23005 webserver.cc:460] Webserver started at http://127.7.67.190:37999/ using document root <none> and password file <none>
I20260811 05:14:15.219696 23005 fs_manager.cc:359] Metadata directory not provided
I20260811 05:14:15.219770 23005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:14:15.220013 23005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:14:15.222345 23005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "c5c938e9b4934924938592602e343e1f"
format_stamp: "Formatted at 2026-08-11 05:14:15 on dist-test-slave-04bb"
server_key: "511cb4ad31809580e86624d60eaa642f"
server_key_iv: "8cfe69d46f687245ff8cb710c2ce4a28"
server_key_version: "encryptionkey@0"
I20260811 05:14:15.222967 23005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "c5c938e9b4934924938592602e343e1f"
format_stamp: "Formatted at 2026-08-11 05:14:15 on dist-test-slave-04bb"
server_key: "511cb4ad31809580e86624d60eaa642f"
server_key_iv: "8cfe69d46f687245ff8cb710c2ce4a28"
server_key_version: "encryptionkey@0"
I20260811 05:14:15.227087 23005 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 05:14:15.229739 23025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:15.231005 23005 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:14:15.231150 23005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "c5c938e9b4934924938592602e343e1f"
format_stamp: "Formatted at 2026-08-11 05:14:15 on dist-test-slave-04bb"
server_key: "511cb4ad31809580e86624d60eaa642f"
server_key_iv: "8cfe69d46f687245ff8cb710c2ce4a28"
server_key_version: "encryptionkey@0"
I20260811 05:14:15.231268 23005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:14:15.235999 23005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:14:15.236712 23005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:14:15.236899 23005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:14:15.244242 23005 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:40683
I20260811 05:14:15.244261 23105 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:40683 every 8 connection(s)
I20260811 05:14:15.245353 23005 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:14:15.248770 23107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:14:15.254451 23107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5c938e9b4934924938592602e343e1f: Bootstrap starting.
I20260811 05:14:15.254904 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 23005
I20260811 05:14:15.255024 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:14:15.255311 7438 external_mini_cluster.cc:1428] Setting key 7b369e871baabfaac24c0efc24804e05
I20260811 05:14:15.257349 23107 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5c938e9b4934924938592602e343e1f: Neither blocks nor log segments found. Creating new log.
I20260811 05:14:15.258267 23107 log.cc:824] T 00000000000000000000000000000000 P c5c938e9b4934924938592602e343e1f: Log is configured to *not* fsync() on all Append() calls
I20260811 05:14:15.260668 23107 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5c938e9b4934924938592602e343e1f: No bootstrap required, opened a new log
I20260811 05:14:15.261662 23110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:15.256809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58022 (local address 127.7.67.190:40683)
0811 05:14:15.257582 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:15.257606 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:15.257655 (+ 49us) server_negotiation.cc:407] Connection header received
0811 05:14:15.258520 (+ 865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:15.258547 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:15.258918 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:15.259314 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:15.260075 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:14:15.260081 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:14:15.260096 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:14:15.260108 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:14:15.260184 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:14:15.260207 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:14:15.260378 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:15.260449 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 05:14:15.260723 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":145,"threads_started":1}
I20260811 05:14:15.265009 23107 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5c938e9b4934924938592602e343e1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c938e9b4934924938592602e343e1f" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 40683 } }
I20260811 05:14:15.265225 23107 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5c938e9b4934924938592602e343e1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:14:15.265260 23107 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5c938e9b4934924938592602e343e1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5c938e9b4934924938592602e343e1f, State: Initialized, Role: FOLLOWER
I20260811 05:14:15.265671 23107 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5c938e9b4934924938592602e343e1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c938e9b4934924938592602e343e1f" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 40683 } }
I20260811 05:14:15.265800 23107 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5c938e9b4934924938592602e343e1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:14:15.265847 23107 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5c938e9b4934924938592602e343e1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:14:15.265919 23107 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5c938e9b4934924938592602e343e1f [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:14:15.266799 23107 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5c938e9b4934924938592602e343e1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c938e9b4934924938592602e343e1f" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 40683 } }
I20260811 05:14:15.267087 23107 leader_election.cc:302] T 00000000000000000000000000000000 P c5c938e9b4934924938592602e343e1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5c938e9b4934924938592602e343e1f; no voters:
I20260811 05:14:15.267339 23107 leader_election.cc:288] T 00000000000000000000000000000000 P c5c938e9b4934924938592602e343e1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:14:15.267524 23113 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5c938e9b4934924938592602e343e1f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:14:15.267813 23113 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5c938e9b4934924938592602e343e1f [term 1 LEADER]: Becoming Leader. State: Replica: c5c938e9b4934924938592602e343e1f, State: Running, Role: LEADER
I20260811 05:14:15.268204 23113 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5c938e9b4934924938592602e343e1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c938e9b4934924938592602e343e1f" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 40683 } }
I20260811 05:14:15.268774 23107 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5c938e9b4934924938592602e343e1f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:14:15.270187 23115 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5c938e9b4934924938592602e343e1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5c938e9b4934924938592602e343e1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c938e9b4934924938592602e343e1f" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 40683 } } }
I20260811 05:14:15.270362 23115 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5c938e9b4934924938592602e343e1f [sys.catalog]: This master's current role is: LEADER
I20260811 05:14:15.270187 23122 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5c938e9b4934924938592602e343e1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5c938e9b4934924938592602e343e1f. Latest consensus state: current_term: 1 leader_uuid: "c5c938e9b4934924938592602e343e1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5c938e9b4934924938592602e343e1f" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 40683 } } }
I20260811 05:14:15.270432 23122 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5c938e9b4934924938592602e343e1f [sys.catalog]: This master's current role is: LEADER
I20260811 05:14:15.270838 23140 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:14:15.273782 23140 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:14:15.279588 23140 catalog_manager.cc:1347] Generated new cluster ID: f707826bf4ce4056b453bf4348b81565
I20260811 05:14:15.279671 23140 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:14:15.301388 23140 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:14:15.301759 23140 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:14:15.312104 23140 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5c938e9b4934924938592602e343e1f: Generated new TSK 0
I20260811 05:14:15.312851 23140 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:14:15.322088 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:40683
--builtin_ntp_servers=127.7.67.148:44151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 05:14:15.438398 23152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:14:15.438647 23152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:14:15.438732 23152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:14:15.442833 23152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:14:15.442893 23152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:14:15.442983 23152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:14:15.447839 23152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:44151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:40683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:14:15.449029 23152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:14:15.450456 23152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:14:15.460342 23166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:15.460355 23162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:15.460461 23163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:14:15.460671 23152 server_base.cc:1034] running on GCE node
I20260811 05:14:15.461017 23152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:14:15.461565 23152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:14:15.462796 23152 hybrid_clock.cc:648] HybridClock initialized: now 1786425255462782 us; error 31 us; skew 500 ppm
I20260811 05:14:15.464942 23152 webserver.cc:460] Webserver started at http://127.7.67.129:37339/ using document root <none> and password file <none>
I20260811 05:14:15.465541 23152 fs_manager.cc:359] Metadata directory not provided
I20260811 05:14:15.465639 23152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:14:15.465859 23152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:14:15.467935 23152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "13c09d24515846d4883c2c478c59a06e"
format_stamp: "Formatted at 2026-08-11 05:14:15 on dist-test-slave-04bb"
server_key: "1b2f67759170371517bbec109efa4486"
server_key_iv: "ca1e31d2a210dea072fa6ed4580a1c6e"
server_key_version: "encryptionkey@0"
I20260811 05:14:15.468561 23152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "13c09d24515846d4883c2c478c59a06e"
format_stamp: "Formatted at 2026-08-11 05:14:15 on dist-test-slave-04bb"
server_key: "1b2f67759170371517bbec109efa4486"
server_key_iv: "ca1e31d2a210dea072fa6ed4580a1c6e"
server_key_version: "encryptionkey@0"
I20260811 05:14:15.472755 23152 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 05:14:15.475301 23175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:15.476784 23152 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:14:15.476930 23152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "13c09d24515846d4883c2c478c59a06e"
format_stamp: "Formatted at 2026-08-11 05:14:15 on dist-test-slave-04bb"
server_key: "1b2f67759170371517bbec109efa4486"
server_key_iv: "ca1e31d2a210dea072fa6ed4580a1c6e"
server_key_version: "encryptionkey@0"
I20260811 05:14:15.477053 23152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:14:15.481901 23152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:14:15.482544 23152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:14:15.482740 23152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:14:15.483310 23152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:14:15.484241 23152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:14:15.484316 23152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:15.484382 23152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:14:15.484426 23152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:15.493873 23152 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:44579
I20260811 05:14:15.493898 23342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:44579 every 8 connection(s)
I20260811 05:14:15.494824 23152 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:14:15.499644 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 23152
I20260811 05:14:15.499740 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:14:15.500038 7438 external_mini_cluster.cc:1428] Setting key 31054d5fbb5a1d3f3d91c63ab4d06eac
I20260811 05:14:15.501997 23110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:15.496968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:47467 (local address 127.7.67.190:40683)
0811 05:14:15.497116 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:15.497121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:15.498599 (+ 1478us) server_negotiation.cc:407] Connection header received
0811 05:14:15.500194 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:15.500198 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:15.500245 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:15.500329 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:15.501505 (+ 1176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:14:15.501509 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:14:15.501512 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:14:15.501525 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:14:15.501548 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:14:15.501559 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:14:15.501670 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:15.501823 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 05:14:15.501869 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 05:14:15.502161 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:40683
--builtin_ntp_servers=127.7.67.148:44151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 05:14:15.502841 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:15.497474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:40683 (local address 127.7.67.129:47467)
0811 05:14:15.498427 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:15.498489 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:15.499975 (+ 1486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:15.500498 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:15.500505 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:15.500652 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:15.501250 (+ 598us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:14:15.501251 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:15.501290 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:14:15.501356 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:14:15.501651 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:14:15.501660 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:14:15.501667 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 05:14:15.501854 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 05:14:15.502093 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":166,"threads_started":1}
I20260811 05:14:15.504144 23344 heartbeater.cc:347] Connected to a master server at 127.7.67.190:40683
I20260811 05:14:15.504376 23344 heartbeater.cc:464] Registering TS with master...
I20260811 05:14:15.504830 23344 heartbeater.cc:511] Master 127.7.67.190:40683 requested a full tablet report, sending...
I20260811 05:14:15.506435 23049 ts_manager.cc:194] Registered new tserver with Master: 13c09d24515846d4883c2c478c59a06e (127.7.67.129:44579)
W20260811 05:14:15.619535 23347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:14:15.619829 23347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:14:15.619903 23347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:14:15.623963 23347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:14:15.624045 23347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:14:15.624135 23347 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:14:15.628885 23347 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:44151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:40683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:14:15.629992 23347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:14:15.631459 23347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:14:15.641408 23353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:15.641408 23354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:14:15.641572 23347 server_base.cc:1034] running on GCE node
W20260811 05:14:15.641783 23356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:14:15.642192 23347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:14:15.642820 23347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:14:15.643999 23347 hybrid_clock.cc:648] HybridClock initialized: now 1786425255643981 us; error 28 us; skew 500 ppm
I20260811 05:14:15.645946 23347 webserver.cc:460] Webserver started at http://127.7.67.130:36031/ using document root <none> and password file <none>
I20260811 05:14:15.646543 23347 fs_manager.cc:359] Metadata directory not provided
I20260811 05:14:15.646605 23347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:14:15.646833 23347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:14:15.649021 23347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "618241e50cac4c4685ed42f9450df864"
format_stamp: "Formatted at 2026-08-11 05:14:15 on dist-test-slave-04bb"
server_key: "dcf4c89df04ec95320779d8267d6bdd2"
server_key_iv: "14363ad6835f39bedd1e8a294a3539f0"
server_key_version: "encryptionkey@0"
I20260811 05:14:15.649585 23347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "618241e50cac4c4685ed42f9450df864"
format_stamp: "Formatted at 2026-08-11 05:14:15 on dist-test-slave-04bb"
server_key: "dcf4c89df04ec95320779d8267d6bdd2"
server_key_iv: "14363ad6835f39bedd1e8a294a3539f0"
server_key_version: "encryptionkey@0"
I20260811 05:14:15.653507 23347 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 05:14:15.656021 23365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:15.657382 23347 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:14:15.657519 23347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "618241e50cac4c4685ed42f9450df864"
format_stamp: "Formatted at 2026-08-11 05:14:15 on dist-test-slave-04bb"
server_key: "dcf4c89df04ec95320779d8267d6bdd2"
server_key_iv: "14363ad6835f39bedd1e8a294a3539f0"
server_key_version: "encryptionkey@0"
I20260811 05:14:15.657634 23347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:14:15.662333 23347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:14:15.662955 23347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:14:15.663137 23347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:14:15.663686 23347 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:14:15.664687 23347 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:14:15.664754 23347 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:15.664819 23347 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:14:15.664866 23347 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:15.674415 23347 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:34887
I20260811 05:14:15.674484 23548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:34887 every 8 connection(s)
I20260811 05:14:15.675395 23347 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:14:15.681815 23110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:15.677339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:57381 (local address 127.7.67.190:40683)
0811 05:14:15.677514 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:15.677519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:15.678567 (+ 1048us) server_negotiation.cc:407] Connection header received
0811 05:14:15.679884 (+ 1317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:15.679887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:15.679925 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:15.680023 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:15.681383 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:14:15.681386 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:14:15.681388 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:14:15.681403 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:14:15.681425 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:14:15.681434 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:14:15.681533 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:15.681661 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 05:14:15.681703 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 05:14:15.682310 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 23347
I20260811 05:14:15.682454 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:14:15.682758 7438 external_mini_cluster.cc:1428] Setting key f6dee2b7da64e3790a5db7a84dfc97f8
I20260811 05:14:15.682782 23552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:15.677737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:40683 (local address 127.7.67.130:57381)
0811 05:14:15.678412 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:15.678452 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:15.679660 (+ 1208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:15.680256 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:15.680263 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:15.680360 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:15.681026 (+ 666us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:14:15.681028 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:15.681070 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:14:15.681139 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:14:15.681531 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:14:15.681539 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:14:15.681543 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:14:15.681681 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 05:14:15.681880 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":104,"threads_started":1}
I20260811 05:14:15.684074 23550 heartbeater.cc:347] Connected to a master server at 127.7.67.190:40683
I20260811 05:14:15.684316 23550 heartbeater.cc:464] Registering TS with master...
I20260811 05:14:15.684799 23550 heartbeater.cc:511] Master 127.7.67.190:40683 requested a full tablet report, sending...
I20260811 05:14:15.685114 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:40683
--builtin_ntp_servers=127.7.67.148:44151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 05:14:15.685964 23048 ts_manager.cc:194] Registered new tserver with Master: 618241e50cac4c4685ed42f9450df864 (127.7.67.130:34887)
W20260811 05:14:15.799922 23554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:14:15.800195 23554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:14:15.800235 23554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:14:15.804265 23554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:14:15.804322 23554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:14:15.804411 23554 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:14:15.809226 23554 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:44151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:40683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:14:15.810460 23554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:14:15.811892 23554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:14:15.821653 23566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:15.821638 23570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:15.821655 23567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:14:15.822014 23554 server_base.cc:1034] running on GCE node
I20260811 05:14:15.822415 23554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:14:15.823065 23554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:14:15.824234 23554 hybrid_clock.cc:648] HybridClock initialized: now 1786425255824214 us; error 30 us; skew 500 ppm
I20260811 05:14:15.826122 23554 webserver.cc:460] Webserver started at http://127.7.67.131:38681/ using document root <none> and password file <none>
I20260811 05:14:15.826732 23554 fs_manager.cc:359] Metadata directory not provided
I20260811 05:14:15.826795 23554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:14:15.827051 23554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:14:15.829193 23554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "af29326d780c4347ae948a26880549eb"
format_stamp: "Formatted at 2026-08-11 05:14:15 on dist-test-slave-04bb"
server_key: "70de4b5b5b5cb33a627ddb3511e5e408"
server_key_iv: "94c7f3534755426c2b6918dbd6c10098"
server_key_version: "encryptionkey@0"
I20260811 05:14:15.829758 23554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "af29326d780c4347ae948a26880549eb"
format_stamp: "Formatted at 2026-08-11 05:14:15 on dist-test-slave-04bb"
server_key: "70de4b5b5b5cb33a627ddb3511e5e408"
server_key_iv: "94c7f3534755426c2b6918dbd6c10098"
server_key_version: "encryptionkey@0"
I20260811 05:14:15.833338 23554 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 05:14:15.836107 23580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:15.837595 23554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:14:15.837721 23554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "af29326d780c4347ae948a26880549eb"
format_stamp: "Formatted at 2026-08-11 05:14:15 on dist-test-slave-04bb"
server_key: "70de4b5b5b5cb33a627ddb3511e5e408"
server_key_iv: "94c7f3534755426c2b6918dbd6c10098"
server_key_version: "encryptionkey@0"
I20260811 05:14:15.837837 23554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:14:15.842469 23554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:14:15.843043 23554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:14:15.843178 23554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:14:15.843662 23554 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:14:15.844594 23554 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:14:15.844648 23554 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:15.844688 23554 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:14:15.844712 23554 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:15.853425 23554 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:40027
I20260811 05:14:15.853449 23760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:40027 every 8 connection(s)
I20260811 05:14:15.854329 23554 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:14:15.860937 23110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:15.856300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:42365 (local address 127.7.67.190:40683)
0811 05:14:15.856478 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:15.856483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:15.857674 (+ 1191us) server_negotiation.cc:407] Connection header received
0811 05:14:15.859065 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:15.859068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:15.859107 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:15.859185 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:15.860408 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:14:15.860411 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:14:15.860413 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:14:15.860424 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:14:15.860444 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:14:15.860453 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:14:15.860585 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:15.860787 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 05:14:15.860829 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 05:14:15.861995 23765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:15.856775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:40683 (local address 127.7.67.131:42365)
0811 05:14:15.857515 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:15.857560 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:15.858851 (+ 1291us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:15.859370 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:15.859377 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:15.859475 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:15.860138 (+ 663us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:14:15.860140 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:15.860182 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:14:15.860261 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:14:15.860593 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:14:15.860605 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:14:15.860616 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 05:14:15.860821 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 05:14:15.861132 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":120,"threads_started":1}
I20260811 05:14:15.862689 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 23554
I20260811 05:14:15.862825 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:14:15.863179 7438 external_mini_cluster.cc:1428] Setting key 5af46171717699104857f11f3bcfce22
I20260811 05:14:15.863380 23761 heartbeater.cc:347] Connected to a master server at 127.7.67.190:40683
I20260811 05:14:15.863651 23761 heartbeater.cc:464] Registering TS with master...
I20260811 05:14:15.864143 23761 heartbeater.cc:511] Master 127.7.67.190:40683 requested a full tablet report, sending...
I20260811 05:14:15.865356 23049 ts_manager.cc:194] Registered new tserver with Master: af29326d780c4347ae948a26880549eb (127.7.67.131:40027)
I20260811 05:14:15.877458 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 05:14:15.880698 23110 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:14:15.879595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38786 (local address 127.7.67.190:40683)
0811 05:14:15.879809 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:15.879813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:15.879899 (+ 86us) server_negotiation.cc:407] Connection header received
0811 05:14:15.880008 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:15.880010 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:15.880045 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:15.880114 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:15.880486 (+ 372us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38786: BlockingRecv error: recv got EOF from 127.0.0.1:38786 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
I20260811 05:14:15.881022 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 23152
I20260811 05:14:15.896746 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 23347
I20260811 05:14:15.906104 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 23554
I20260811 05:14:15.919226 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 23005
2026-08-11T05:14:15Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (874 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:14:15 dist-test-slave-04bb krb5kdc[23777](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:14:15 dist-test-slave-04bb krb5kdc[23777](info): set up 2 sockets
Aug 11 05:14:15 dist-test-slave-04bb krb5kdc[23777](info): commencing operation
krb5kdc: starting...
W20260811 05:14:17.188244 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:14:17 dist-test-slave-04bb krb5kdc[23777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:14:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:14:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:14:17.319813 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:39295
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:44675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:39295
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:14:17.436589 23799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:14:17.436909 23799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:14:17.436993 23799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:14:17.440995 23799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:14:17.441048 23799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:14:17.441068 23799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:14:17.441085 23799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:14:17.441100 23799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:14:17.446403 23799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:44675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:39295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:39295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:14:17.447511 23799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:14:17.448581 23799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:14:17.457100 23799 server_base.cc:1034] running on GCE node
W20260811 05:14:17.457141 23809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:17.457180 23812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:17.457319 23815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:14:17.457937 23799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:14:17.459043 23799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:14:17.460273 23799 hybrid_clock.cc:648] HybridClock initialized: now 1786425257460240 us; error 41 us; skew 500 ppm
Aug 11 05:14:17 dist-test-slave-04bb krb5kdc[23777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425257, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:14:17.463485 23799 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:14:17.464457 23799 webserver.cc:460] Webserver started at http://127.7.67.190:42705/ using document root <none> and password file <none>
I20260811 05:14:17.465083 23799 fs_manager.cc:359] Metadata directory not provided
I20260811 05:14:17.465183 23799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:14:17.465433 23799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:14:17.467653 23799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "7975cf7d9bb146a6846758eea8ff03d3"
format_stamp: "Formatted at 2026-08-11 05:14:17 on dist-test-slave-04bb"
server_key: "fc5fc8ecc9bc1bdd1654098d9a6efbc5"
server_key_iv: "9389b78d9f25b7f6fe4d8fe85f7e12d0"
server_key_version: "encryptionkey@0"
I20260811 05:14:17.468262 23799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "7975cf7d9bb146a6846758eea8ff03d3"
format_stamp: "Formatted at 2026-08-11 05:14:17 on dist-test-slave-04bb"
server_key: "fc5fc8ecc9bc1bdd1654098d9a6efbc5"
server_key_iv: "9389b78d9f25b7f6fe4d8fe85f7e12d0"
server_key_version: "encryptionkey@0"
I20260811 05:14:17.472268 23799 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 05:14:17.474965 23824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:17.476197 23799 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:14:17.476346 23799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "7975cf7d9bb146a6846758eea8ff03d3"
format_stamp: "Formatted at 2026-08-11 05:14:17 on dist-test-slave-04bb"
server_key: "fc5fc8ecc9bc1bdd1654098d9a6efbc5"
server_key_iv: "9389b78d9f25b7f6fe4d8fe85f7e12d0"
server_key_version: "encryptionkey@0"
I20260811 05:14:17.476460 23799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:14:17.492057 23799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:14:17.495436 23799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:14:17.495651 23799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:14:17.503520 23799 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:39295
I20260811 05:14:17.503875 23900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:39295 every 8 connection(s)
I20260811 05:14:17.504873 23799 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:14:17.508121 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 23799
I20260811 05:14:17.508168 23901 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:14:17.508270 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:14:17.508625 7438 external_mini_cluster.cc:1428] Setting key d675e2c6e39631f73c7e23a7b044d1ef
I20260811 05:14:17.515463 23901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7975cf7d9bb146a6846758eea8ff03d3: Bootstrap starting.
Aug 11 05:14:17 dist-test-slave-04bb krb5kdc[23777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425257, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:14:17.518491 23901 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7975cf7d9bb146a6846758eea8ff03d3: Neither blocks nor log segments found. Creating new log.
I20260811 05:14:17.519265 23901 log.cc:824] T 00000000000000000000000000000000 P 7975cf7d9bb146a6846758eea8ff03d3: Log is configured to *not* fsync() on all Append() calls
I20260811 05:14:17.521497 23901 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7975cf7d9bb146a6846758eea8ff03d3: No bootstrap required, opened a new log
I20260811 05:14:17.524232 23904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:17.510153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42612 (local address 127.7.67.190:39295)
0811 05:14:17.510564 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:17.510582 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:17.510630 (+ 48us) server_negotiation.cc:407] Connection header received
0811 05:14:17.511498 (+ 868us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:17.511518 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:17.511866 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:17.512146 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:17.513134 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:17.514165 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:17.515333 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:17.515882 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:17.516007 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:17.518954 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:14:17.518959 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:14:17.518973 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:14:17.519002 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:14:17.521125 (+ 2123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:14:17.521676 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:14:17.521683 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:14:17.521695 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:14:17.521816 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:14:17.522157 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:14:17.522191 (+ 34us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:14:17.522193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:14:17.522612 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:14:17.522775 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:17.523117 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 05:14:17.523496 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":133,"threads_started":1}
I20260811 05:14:17.525664 23901 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7975cf7d9bb146a6846758eea8ff03d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7975cf7d9bb146a6846758eea8ff03d3" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39295 } }
I20260811 05:14:17.525951 23901 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7975cf7d9bb146a6846758eea8ff03d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:14:17.526024 23901 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7975cf7d9bb146a6846758eea8ff03d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7975cf7d9bb146a6846758eea8ff03d3, State: Initialized, Role: FOLLOWER
I20260811 05:14:17.526485 23901 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7975cf7d9bb146a6846758eea8ff03d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7975cf7d9bb146a6846758eea8ff03d3" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39295 } }
I20260811 05:14:17.526669 23901 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7975cf7d9bb146a6846758eea8ff03d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:14:17.526746 23901 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7975cf7d9bb146a6846758eea8ff03d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:14:17.526870 23901 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7975cf7d9bb146a6846758eea8ff03d3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:14:17.527740 23901 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7975cf7d9bb146a6846758eea8ff03d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7975cf7d9bb146a6846758eea8ff03d3" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39295 } }
I20260811 05:14:17.528045 23901 leader_election.cc:302] T 00000000000000000000000000000000 P 7975cf7d9bb146a6846758eea8ff03d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7975cf7d9bb146a6846758eea8ff03d3; no voters:
I20260811 05:14:17.528312 23901 leader_election.cc:288] T 00000000000000000000000000000000 P 7975cf7d9bb146a6846758eea8ff03d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:14:17.528450 23906 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7975cf7d9bb146a6846758eea8ff03d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:14:17.528743 23906 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7975cf7d9bb146a6846758eea8ff03d3 [term 1 LEADER]: Becoming Leader. State: Replica: 7975cf7d9bb146a6846758eea8ff03d3, State: Running, Role: LEADER
I20260811 05:14:17.529052 23906 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7975cf7d9bb146a6846758eea8ff03d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7975cf7d9bb146a6846758eea8ff03d3" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39295 } }
I20260811 05:14:17.529619 23901 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7975cf7d9bb146a6846758eea8ff03d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:14:17.530920 23908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7975cf7d9bb146a6846758eea8ff03d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7975cf7d9bb146a6846758eea8ff03d3. Latest consensus state: current_term: 1 leader_uuid: "7975cf7d9bb146a6846758eea8ff03d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7975cf7d9bb146a6846758eea8ff03d3" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39295 } } }
I20260811 05:14:17.530928 23907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7975cf7d9bb146a6846758eea8ff03d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7975cf7d9bb146a6846758eea8ff03d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7975cf7d9bb146a6846758eea8ff03d3" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39295 } } }
I20260811 05:14:17.531057 23908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7975cf7d9bb146a6846758eea8ff03d3 [sys.catalog]: This master's current role is: LEADER
I20260811 05:14:17.531057 23907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7975cf7d9bb146a6846758eea8ff03d3 [sys.catalog]: This master's current role is: LEADER
I20260811 05:14:17.531678 23918 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:14:17.534528 23918 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:14:17.539803 23918 catalog_manager.cc:1347] Generated new cluster ID: 0f6b145f206c457f982d0bd93c463a53
I20260811 05:14:17.539868 23918 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:14:17.545907 23918 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:14:17.547088 23918 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:14:17.553232 23918 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7975cf7d9bb146a6846758eea8ff03d3: Generated new TSK 0
I20260811 05:14:17.553910 23918 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
I20260811 05:14:17.591678 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:39295
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:44675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:14:17.721334 23936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:14:17.721575 23936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:14:17.721616 23936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:14:17.725473 23936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:14:17.725520 23936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:14:17.725575 23936 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:14:17.730918 23936 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:44675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:39295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:14:17.731904 23936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:14:17.732908 23936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:14:17.742799 23941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:14:17.742841 23936 server_base.cc:1034] running on GCE node
W20260811 05:14:17.742753 23942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:17.742997 23945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:14:17.743383 23936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:14:17.744026 23936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:14:17.745203 23936 hybrid_clock.cc:648] HybridClock initialized: now 1786425257745168 us; error 49 us; skew 500 ppm
Aug 11 05:14:17 dist-test-slave-04bb krb5kdc[23777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425257, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:14:17.748057 23936 init.cc:377] Logged in from keytab as kudu/127.7.67.129@KRBTEST.COM (short username kudu)
I20260811 05:14:17.748953 23936 webserver.cc:460] Webserver started at http://127.7.67.129:41803/ using document root <none> and password file <none>
I20260811 05:14:17.749483 23936 fs_manager.cc:359] Metadata directory not provided
I20260811 05:14:17.749543 23936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:14:17.749727 23936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:14:17.751703 23936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "b58ec5338af94b31967bbd1eed5b38ee"
format_stamp: "Formatted at 2026-08-11 05:14:17 on dist-test-slave-04bb"
server_key: "14200483206932397d5ea46afdde176d"
server_key_iv: "8917419ef79775284d7f4aaeac7d3464"
server_key_version: "encryptionkey@0"
I20260811 05:14:17.752224 23936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "b58ec5338af94b31967bbd1eed5b38ee"
format_stamp: "Formatted at 2026-08-11 05:14:17 on dist-test-slave-04bb"
server_key: "14200483206932397d5ea46afdde176d"
server_key_iv: "8917419ef79775284d7f4aaeac7d3464"
server_key_version: "encryptionkey@0"
I20260811 05:14:17.756012 23936 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 05:14:17.758562 23960 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:17.759833 23936 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:14:17.759955 23936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "b58ec5338af94b31967bbd1eed5b38ee"
format_stamp: "Formatted at 2026-08-11 05:14:17 on dist-test-slave-04bb"
server_key: "14200483206932397d5ea46afdde176d"
server_key_iv: "8917419ef79775284d7f4aaeac7d3464"
server_key_version: "encryptionkey@0"
I20260811 05:14:17.760064 23936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:14:17.785177 23936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:14:17.788599 23936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:14:17.788818 23936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:14:17.789507 23936 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:14:17.790370 23936 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:14:17.790417 23936 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:17.790457 23936 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:14:17.790472 23936 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:17.799432 23936 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:43489
I20260811 05:14:17.799461 24151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:43489 every 8 connection(s)
I20260811 05:14:17.800349 23936 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:14:17.809708 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 23936
I20260811 05:14:17.809840 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
Aug 11 05:14:17 dist-test-slave-04bb krb5kdc[23777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425257, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:14:17.810154 7438 external_mini_cluster.cc:1428] Setting key 3e0a2ea90a43181357748e40d7f43d47
I20260811 05:14:17.814486 23904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:17.802411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:41811 (local address 127.7.67.190:39295)
0811 05:14:17.802547 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:17.802550 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:17.803416 (+ 866us) server_negotiation.cc:407] Connection header received
0811 05:14:17.804772 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:17.804775 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:17.804825 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:17.804899 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:17.805990 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:17.806585 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:17.807664 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:17.807894 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:17.807972 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:17.810692 (+ 2720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:14:17.810696 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:14:17.810699 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:14:17.810733 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:14:17.812324 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:14:17.812920 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:14:17.812924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:14:17.812926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:14:17.813017 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:14:17.813379 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:14:17.813383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:14:17.813385 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:14:17.813573 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:14:17.813692 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:17.814182 (+ 490us) server_negotiation.cc:299] Negotiation successful
0811 05:14:17.814314 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 05:14:17.815279 24154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:17.802796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:39295 (local address 127.7.67.129:41811)
0811 05:14:17.803232 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:17.803302 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:17.804512 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:17.805027 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:17.805034 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:17.805382 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:17.805848 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:17.805861 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:17.806696 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:17.806698 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:17.807560 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:17.807568 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:17.808023 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:17.808026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:17.808165 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:17.808744 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:17.808760 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:17.810509 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:14:17.812460 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:14:17.812466 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:14:17.812480 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:14:17.812807 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:14:17.813112 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:14:17.813116 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:14:17.813118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:14:17.813271 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:14:17.813682 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:14:17.813689 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:14:17.813937 (+ 248us) client_negotiation.cc:769] Sending connection context
0811 05:14:17.814121 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 05:14:17.814450 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":118,"threads_started":1}
I20260811 05:14:17.816726 24152 heartbeater.cc:347] Connected to a master server at 127.7.67.190:39295
I20260811 05:14:17.816979 24152 heartbeater.cc:464] Registering TS with master...
I20260811 05:14:17.817427 24152 heartbeater.cc:511] Master 127.7.67.190:39295 requested a full tablet report, sending...
I20260811 05:14:17.819095 23846 ts_manager.cc:194] Registered new tserver with Master: b58ec5338af94b31967bbd1eed5b38ee (127.7.67.129:43489)
I20260811 05:14:17.820832 23846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.129@KRBTEST.COM'} at 127.7.67.129:41811
WARNING: no policy specified for kudu/127.7.67.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
I20260811 05:14:17.839757 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:39295
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:44675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:14:17.954839 24157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:14:17.955084 24157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:14:17.955125 24157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:14:17.959112 24157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:14:17.959168 24157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:14:17.959268 24157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:14:17.964426 24157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:44675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:39295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:14:17.965467 24157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:14:17.966553 24157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:14:17.976630 24168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:17.976636 24165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:14:17.976691 24157 server_base.cc:1034] running on GCE node
W20260811 05:14:17.976636 24164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:14:17.977309 24157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:14:17.977958 24157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:14:17.979192 24157 hybrid_clock.cc:648] HybridClock initialized: now 1786425257979152 us; error 24 us; skew 500 ppm
Aug 11 05:14:17 dist-test-slave-04bb krb5kdc[23777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425257, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:14:17.982144 24157 init.cc:377] Logged in from keytab as kudu/127.7.67.130@KRBTEST.COM (short username kudu)
I20260811 05:14:17.983081 24157 webserver.cc:460] Webserver started at http://127.7.67.130:33973/ using document root <none> and password file <none>
I20260811 05:14:17.983664 24157 fs_manager.cc:359] Metadata directory not provided
I20260811 05:14:17.983757 24157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:14:17.983996 24157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:14:17.986095 24157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "4d364d73a3d343b69e1b6595399dbc8c"
format_stamp: "Formatted at 2026-08-11 05:14:17 on dist-test-slave-04bb"
server_key: "dbc156417152ab1c273a23c66ffe5f4b"
server_key_iv: "5a0223a9e32878e4dd67a841a1223373"
server_key_version: "encryptionkey@0"
I20260811 05:14:17.986689 24157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "4d364d73a3d343b69e1b6595399dbc8c"
format_stamp: "Formatted at 2026-08-11 05:14:17 on dist-test-slave-04bb"
server_key: "dbc156417152ab1c273a23c66ffe5f4b"
server_key_iv: "5a0223a9e32878e4dd67a841a1223373"
server_key_version: "encryptionkey@0"
I20260811 05:14:17.990826 24157 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 05:14:17.993403 24179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:17.994602 24157 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:14:17.994747 24157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "4d364d73a3d343b69e1b6595399dbc8c"
format_stamp: "Formatted at 2026-08-11 05:14:17 on dist-test-slave-04bb"
server_key: "dbc156417152ab1c273a23c66ffe5f4b"
server_key_iv: "5a0223a9e32878e4dd67a841a1223373"
server_key_version: "encryptionkey@0"
I20260811 05:14:17.994858 24157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:14:18.004835 24157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:14:18.008093 24157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:14:18.008301 24157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:14:18.008970 24157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:14:18.009816 24157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:14:18.009891 24157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:18.009958 24157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:14:18.010010 24157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:18.018559 24157 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:36087
I20260811 05:14:18.018586 24362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:36087 every 8 connection(s)
I20260811 05:14:18.019457 24157 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:14:18.026612 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 24157
I20260811 05:14:18.026757 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:14:18.027112 7438 external_mini_cluster.cc:1428] Setting key f1eb7c6b5b7881360d1009ec45d47561
Aug 11 05:14:18 dist-test-slave-04bb krb5kdc[23777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425257, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:14:18.034590 23904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:18.021425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:45331 (local address 127.7.67.190:39295)
0811 05:14:18.021561 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:18.021565 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:18.022376 (+ 811us) server_negotiation.cc:407] Connection header received
0811 05:14:18.023764 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:18.023768 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:18.023828 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:18.023936 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:18.025192 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.025719 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.026934 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.027322 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.027451 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:18.030534 (+ 3083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:14:18.030538 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:14:18.030541 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:14:18.030569 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:14:18.032422 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:14:18.032993 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:14:18.032997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:14:18.032999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:14:18.033097 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:14:18.033479 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:14:18.033482 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:14:18.033484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:14:18.033674 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:14:18.033785 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:18.034296 (+ 511us) server_negotiation.cc:299] Negotiation successful
0811 05:14:18.034419 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 05:14:18.035442 24368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:18.021768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:39295 (local address 127.7.67.130:45331)
0811 05:14:18.022228 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:18.022272 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:18.023547 (+ 1275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:18.024114 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:18.024121 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:18.024477 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:18.025035 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:18.025048 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.025854 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.025862 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:18.026762 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:18.026773 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.027485 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.027488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:18.027750 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:18.028420 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:18.028437 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:18.030352 (+ 1915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:14:18.032577 (+ 2225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:14:18.032583 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:14:18.032599 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:14:18.032880 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:14:18.033216 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:14:18.033219 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:14:18.033221 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:14:18.033369 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:14:18.033786 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:14:18.033794 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:14:18.034042 (+ 248us) client_negotiation.cc:769] Sending connection context
0811 05:14:18.034232 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 05:14:18.034559 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":108,"threads_started":1}
I20260811 05:14:18.036759 24364 heartbeater.cc:347] Connected to a master server at 127.7.67.190:39295
I20260811 05:14:18.037021 24364 heartbeater.cc:464] Registering TS with master...
I20260811 05:14:18.037524 24364 heartbeater.cc:511] Master 127.7.67.190:39295 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.67.131@KRBTEST.COM; defaulting to no policy
I20260811 05:14:18.038681 23846 ts_manager.cc:194] Registered new tserver with Master: 4d364d73a3d343b69e1b6595399dbc8c (127.7.67.130:36087)
I20260811 05:14:18.039268 23846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.130@KRBTEST.COM'} at 127.7.67.130:45331
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
I20260811 05:14:18.056000 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:39295
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:44675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:14:18.171093 24373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:14:18.171411 24373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:14:18.171489 24373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:14:18.175496 24373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:14:18.175550 24373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:14:18.175640 24373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:14:18.180801 24373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:44675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:39295
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:14:18.181785 24373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:14:18.182823 24373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:14:18.192474 24381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:14:18.192500 24380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:14:18.192569 24373 server_base.cc:1034] running on GCE node
W20260811 05:14:18.192499 24384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:14:18.193207 24373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:14:18.193851 24373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:14:18.195053 24373 hybrid_clock.cc:648] HybridClock initialized: now 1786425258195040 us; error 27 us; skew 500 ppm
Aug 11 05:14:18 dist-test-slave-04bb krb5kdc[23777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425258, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:14:18.197983 24373 init.cc:377] Logged in from keytab as kudu/127.7.67.131@KRBTEST.COM (short username kudu)
I20260811 05:14:18.198911 24373 webserver.cc:460] Webserver started at http://127.7.67.131:46431/ using document root <none> and password file <none>
I20260811 05:14:18.199539 24373 fs_manager.cc:359] Metadata directory not provided
I20260811 05:14:18.199632 24373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:14:18.199882 24373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:14:18.202026 24373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "a5648ecf6ec94e41bfa844b04f324f75"
format_stamp: "Formatted at 2026-08-11 05:14:18 on dist-test-slave-04bb"
server_key: "b0905fabed89691b34daa4e1fb8dde81"
server_key_iv: "6caedc0e2b51e92d551ebdaa8cbaf0f7"
server_key_version: "encryptionkey@0"
I20260811 05:14:18.202625 24373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "a5648ecf6ec94e41bfa844b04f324f75"
format_stamp: "Formatted at 2026-08-11 05:14:18 on dist-test-slave-04bb"
server_key: "b0905fabed89691b34daa4e1fb8dde81"
server_key_iv: "6caedc0e2b51e92d551ebdaa8cbaf0f7"
server_key_version: "encryptionkey@0"
I20260811 05:14:18.206489 24373 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 05:14:18.209069 24395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:18.210412 24373 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:14:18.210568 24373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "a5648ecf6ec94e41bfa844b04f324f75"
format_stamp: "Formatted at 2026-08-11 05:14:18 on dist-test-slave-04bb"
server_key: "b0905fabed89691b34daa4e1fb8dde81"
server_key_iv: "6caedc0e2b51e92d551ebdaa8cbaf0f7"
server_key_version: "encryptionkey@0"
I20260811 05:14:18.210691 24373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:14:18.227507 24373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:14:18.231123 24373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:14:18.231341 24373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:14:18.232015 24373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:14:18.232930 24373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:14:18.233002 24373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:18.233076 24373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:14:18.233119 24373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:14:18.241990 24373 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:46873
I20260811 05:14:18.242018 24582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:46873 every 8 connection(s)
I20260811 05:14:18.242913 24373 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
Aug 11 05:14:18 dist-test-slave-04bb krb5kdc[23777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425258, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:14:18.253036 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 24373
I20260811 05:14:18.253149 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:14:18.253502 7438 external_mini_cluster.cc:1428] Setting key 9aba7581c7a343311ef08ecbd1a7f4ab
I20260811 05:14:18.257778 23904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:18.245302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:57485 (local address 127.7.67.190:39295)
0811 05:14:18.245440 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:18.245444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:18.246236 (+ 792us) server_negotiation.cc:407] Connection header received
0811 05:14:18.247589 (+ 1353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:18.247592 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:18.247652 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:18.247763 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:18.248998 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.249548 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.250691 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.250911 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.251020 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:18.253743 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:14:18.253746 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:14:18.253748 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:14:18.253771 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:14:18.255597 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:14:18.256262 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:14:18.256265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:14:18.256266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:14:18.256351 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:14:18.256753 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:14:18.256761 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:14:18.256762 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:14:18.256950 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:14:18.257032 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:18.257544 (+ 512us) server_negotiation.cc:299] Negotiation successful
0811 05:14:18.257645 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 05:14:18.258554 24587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:18.245535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:39295 (local address 127.7.67.131:57485)
0811 05:14:18.246012 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:18.246056 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:18.247371 (+ 1315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:18.247952 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:18.247959 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:18.248312 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:14:18.248850 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:18.248865 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.249696 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.249699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:18.250555 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:18.250565 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.251073 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.251077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:18.251247 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:18.251811 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:14:18.251827 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:14:18.253568 (+ 1741us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:14:18.255738 (+ 2170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:14:18.255752 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:14:18.255769 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:14:18.256121 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:14:18.256444 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:14:18.256446 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:14:18.256448 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:14:18.256658 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:14:18.257052 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:14:18.257058 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:14:18.257289 (+ 231us) client_negotiation.cc:769] Sending connection context
0811 05:14:18.257441 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 05:14:18.257679 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":111,"threads_started":1}
I20260811 05:14:18.259720 24583 heartbeater.cc:347] Connected to a master server at 127.7.67.190:39295
I20260811 05:14:18.259984 24583 heartbeater.cc:464] Registering TS with master...
I20260811 05:14:18.260473 24583 heartbeater.cc:511] Master 127.7.67.190:39295 requested a full tablet report, sending...
I20260811 05:14:18.261588 23846 ts_manager.cc:194] Registered new tserver with Master: a5648ecf6ec94e41bfa844b04f324f75 (127.7.67.131:46873)
I20260811 05:14:18.262127 23846 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.131@KRBTEST.COM'} at 127.7.67.131:57485
I20260811 05:14:18.267911 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:14:18.277786 23904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:18.270027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42614 (local address 127.7.67.190:39295)
0811 05:14:18.270177 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:18.270182 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:18.270395 (+ 213us) server_negotiation.cc:407] Connection header received
0811 05:14:18.270554 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:18.270557 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:18.270602 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:18.270696 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:14:18.271447 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.272043 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.272957 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.273183 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.273291 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:18.274697 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:14:18.274701 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:14:18.274704 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:14:18.274729 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:14:18.276011 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:14:18.276456 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:14:18.276460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:14:18.276462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:14:18.276601 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:14:18.276893 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:14:18.276897 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:14:18.276899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:14:18.277116 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:14:18.277261 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:18.277496 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 05:14:18.277611 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 05:14:18.280622 23846 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:14:18.283726 23846 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:14:18.298439 24601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:18.292343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:36087 (local address 127.0.0.1:42988)
0811 05:14:18.292990 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:18.293041 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:18.293175 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:18.294010 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:18.294019 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:18.294048 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:14:18.294184 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:18.294195 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.295433 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.295438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:18.296740 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:18.296751 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.297870 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.297874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:18.297985 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:18.298028 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 05:14:18.298148 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 05:14:18.298273 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":93,"threads_started":1}
I20260811 05:14:18.298612 24611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:18.293146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33988 (local address 127.7.67.131:46873)
0811 05:14:18.293630 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:18.293658 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:18.293683 (+ 25us) server_negotiation.cc:407] Connection header received
0811 05:14:18.293762 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:18.293767 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:18.293957 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:18.294110 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:14:18.294547 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.295728 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.297157 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.297981 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.298098 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:18.298235 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:18.298353 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 05:14:18.298442 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":165,"threads_started":1}
I20260811 05:14:18.298985 24606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:18.292466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42988 (local address 127.7.67.130:36087)
0811 05:14:18.292850 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:18.292861 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:18.293387 (+ 526us) server_negotiation.cc:407] Connection header received
0811 05:14:18.293473 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:18.293479 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:18.293704 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:18.293868 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:14:18.294356 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.295307 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.296880 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.297689 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.297845 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:18.297994 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:18.298657 (+ 663us) server_negotiation.cc:299] Negotiation successful
0811 05:14:18.298819 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":148,"threads_started":1}
I20260811 05:14:18.298835 24608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:18.293107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:43489 (local address 127.0.0.1:55722)
0811 05:14:18.294135 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:18.294151 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:18.294260 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:18.294778 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:18.294781 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:18.294802 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:14:18.294885 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:18.294891 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.296173 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.296177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:18.297239 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:18.297249 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.298377 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.298380 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:18.298473 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:18.298548 (+ 75us) client_negotiation.cc:769] Sending connection context
0811 05:14:18.298631 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 05:14:18.298708 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":971,"thread_start_us":93,"threads_started":1}
I20260811 05:14:18.299223 24612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:18.293533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55722 (local address 127.7.67.129:43489)
0811 05:14:18.293904 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:18.293922 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:18.294367 (+ 445us) server_negotiation.cc:407] Connection header received
0811 05:14:18.294434 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:18.294437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:18.294621 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:18.294773 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:14:18.295008 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.296042 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.297361 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.298077 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.298171 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:18.298315 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:18.298961 (+ 646us) server_negotiation.cc:299] Negotiation successful
0811 05:14:18.299065 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":84,"threads_started":1}
I20260811 05:14:18.298844 24602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:18.292402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:46873 (local address 127.0.0.1:33988)
0811 05:14:18.293349 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:18.293365 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:18.293458 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:18.294252 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:18.294256 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:18.294274 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:14:18.294406 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:18.294416 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.295872 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.295875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:18.297024 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:18.297033 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.298113 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.298117 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:18.298216 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:18.298239 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 05:14:18.298322 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 05:14:18.298395 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":866,"thread_start_us":135,"threads_started":1}
I20260811 05:14:18.302186 24253 tablet_service.cc:1467] Processing CreateTablet for tablet 48c9078f0730415cabb86e763ee86e35 (DEFAULT_TABLE table=test-table [id=ef408db45cd44d9984163a777077d4c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:14:18.302244 24471 tablet_service.cc:1467] Processing CreateTablet for tablet 48c9078f0730415cabb86e763ee86e35 (DEFAULT_TABLE table=test-table [id=ef408db45cd44d9984163a777077d4c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:14:18.302246 24037 tablet_service.cc:1467] Processing CreateTablet for tablet 48c9078f0730415cabb86e763ee86e35 (DEFAULT_TABLE table=test-table [id=ef408db45cd44d9984163a777077d4c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:14:18.303222 24253 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48c9078f0730415cabb86e763ee86e35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:14:18.303282 24471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48c9078f0730415cabb86e763ee86e35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:14:18.303277 24037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48c9078f0730415cabb86e763ee86e35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:14:18.309706 24613 tablet_bootstrap.cc:492] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c: Bootstrap starting.
I20260811 05:14:18.310791 24615 tablet_bootstrap.cc:492] T 48c9078f0730415cabb86e763ee86e35 P a5648ecf6ec94e41bfa844b04f324f75: Bootstrap starting.
I20260811 05:14:18.312703 24613 tablet_bootstrap.cc:654] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c: Neither blocks nor log segments found. Creating new log.
I20260811 05:14:18.312934 24614 tablet_bootstrap.cc:492] T 48c9078f0730415cabb86e763ee86e35 P b58ec5338af94b31967bbd1eed5b38ee: Bootstrap starting.
I20260811 05:14:18.313001 24615 tablet_bootstrap.cc:654] T 48c9078f0730415cabb86e763ee86e35 P a5648ecf6ec94e41bfa844b04f324f75: Neither blocks nor log segments found. Creating new log.
I20260811 05:14:18.313712 24613 log.cc:824] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c: Log is configured to *not* fsync() on all Append() calls
I20260811 05:14:18.314357 24615 log.cc:824] T 48c9078f0730415cabb86e763ee86e35 P a5648ecf6ec94e41bfa844b04f324f75: Log is configured to *not* fsync() on all Append() calls
I20260811 05:14:18.315044 24614 tablet_bootstrap.cc:654] T 48c9078f0730415cabb86e763ee86e35 P b58ec5338af94b31967bbd1eed5b38ee: Neither blocks nor log segments found. Creating new log.
I20260811 05:14:18.316160 24613 tablet_bootstrap.cc:492] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c: No bootstrap required, opened a new log
I20260811 05:14:18.316311 24614 log.cc:824] T 48c9078f0730415cabb86e763ee86e35 P b58ec5338af94b31967bbd1eed5b38ee: Log is configured to *not* fsync() on all Append() calls
I20260811 05:14:18.316365 24613 ts_tablet_manager.cc:1397] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 05:14:18.316612 24615 tablet_bootstrap.cc:492] T 48c9078f0730415cabb86e763ee86e35 P a5648ecf6ec94e41bfa844b04f324f75: No bootstrap required, opened a new log
I20260811 05:14:18.316793 24615 ts_tablet_manager.cc:1397] T 48c9078f0730415cabb86e763ee86e35 P a5648ecf6ec94e41bfa844b04f324f75: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260811 05:14:18.318041 24614 tablet_bootstrap.cc:492] T 48c9078f0730415cabb86e763ee86e35 P b58ec5338af94b31967bbd1eed5b38ee: No bootstrap required, opened a new log
I20260811 05:14:18.318248 24614 ts_tablet_manager.cc:1397] T 48c9078f0730415cabb86e763ee86e35 P b58ec5338af94b31967bbd1eed5b38ee: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 05:14:18.320294 24613 raft_consensus.cc:348] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d364d73a3d343b69e1b6595399dbc8c" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 36087 } } peers { permanent_uuid: "a5648ecf6ec94e41bfa844b04f324f75" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 46873 } } peers { permanent_uuid: "b58ec5338af94b31967bbd1eed5b38ee" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43489 } }
I20260811 05:14:18.320394 24615 raft_consensus.cc:348] T 48c9078f0730415cabb86e763ee86e35 P a5648ecf6ec94e41bfa844b04f324f75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d364d73a3d343b69e1b6595399dbc8c" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 36087 } } peers { permanent_uuid: "a5648ecf6ec94e41bfa844b04f324f75" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 46873 } } peers { permanent_uuid: "b58ec5338af94b31967bbd1eed5b38ee" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43489 } }
I20260811 05:14:18.320629 24615 raft_consensus.cc:374] T 48c9078f0730415cabb86e763ee86e35 P a5648ecf6ec94e41bfa844b04f324f75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:14:18.320621 24613 raft_consensus.cc:374] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:14:18.320694 24615 raft_consensus.cc:729] T 48c9078f0730415cabb86e763ee86e35 P a5648ecf6ec94e41bfa844b04f324f75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5648ecf6ec94e41bfa844b04f324f75, State: Initialized, Role: FOLLOWER
I20260811 05:14:18.320703 24613 raft_consensus.cc:729] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d364d73a3d343b69e1b6595399dbc8c, State: Initialized, Role: FOLLOWER
I20260811 05:14:18.321033 24614 raft_consensus.cc:348] T 48c9078f0730415cabb86e763ee86e35 P b58ec5338af94b31967bbd1eed5b38ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d364d73a3d343b69e1b6595399dbc8c" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 36087 } } peers { permanent_uuid: "a5648ecf6ec94e41bfa844b04f324f75" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 46873 } } peers { permanent_uuid: "b58ec5338af94b31967bbd1eed5b38ee" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43489 } }
I20260811 05:14:18.321165 24613 consensus_queue.cc:260] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d364d73a3d343b69e1b6595399dbc8c" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 36087 } } peers { permanent_uuid: "a5648ecf6ec94e41bfa844b04f324f75" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 46873 } } peers { permanent_uuid: "b58ec5338af94b31967bbd1eed5b38ee" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43489 } }
I20260811 05:14:18.321166 24615 consensus_queue.cc:260] T 48c9078f0730415cabb86e763ee86e35 P a5648ecf6ec94e41bfa844b04f324f75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d364d73a3d343b69e1b6595399dbc8c" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 36087 } } peers { permanent_uuid: "a5648ecf6ec94e41bfa844b04f324f75" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 46873 } } peers { permanent_uuid: "b58ec5338af94b31967bbd1eed5b38ee" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43489 } }
I20260811 05:14:18.321280 24614 raft_consensus.cc:374] T 48c9078f0730415cabb86e763ee86e35 P b58ec5338af94b31967bbd1eed5b38ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:14:18.321352 24614 raft_consensus.cc:729] T 48c9078f0730415cabb86e763ee86e35 P b58ec5338af94b31967bbd1eed5b38ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b58ec5338af94b31967bbd1eed5b38ee, State: Initialized, Role: FOLLOWER
I20260811 05:14:18.321795 24614 consensus_queue.cc:260] T 48c9078f0730415cabb86e763ee86e35 P b58ec5338af94b31967bbd1eed5b38ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d364d73a3d343b69e1b6595399dbc8c" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 36087 } } peers { permanent_uuid: "a5648ecf6ec94e41bfa844b04f324f75" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 46873 } } peers { permanent_uuid: "b58ec5338af94b31967bbd1eed5b38ee" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43489 } }
I20260811 05:14:18.322155 24613 ts_tablet_manager.cc:1428] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 05:14:18.322155 24615 ts_tablet_manager.cc:1428] T 48c9078f0730415cabb86e763ee86e35 P a5648ecf6ec94e41bfa844b04f324f75: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 05:14:18.322575 24583 heartbeater.cc:503] Master 127.7.67.190:39295 was elected leader, sending a full tablet report...
I20260811 05:14:18.322636 24364 heartbeater.cc:503] Master 127.7.67.190:39295 was elected leader, sending a full tablet report...
I20260811 05:14:18.322772 24614 ts_tablet_manager.cc:1428] T 48c9078f0730415cabb86e763ee86e35 P b58ec5338af94b31967bbd1eed5b38ee: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260811 05:14:18.323104 24152 heartbeater.cc:503] Master 127.7.67.190:39295 was elected leader, sending a full tablet report...
I20260811 05:14:18.407716 24619 raft_consensus.cc:482] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:14:18.407939 24619 raft_consensus.cc:504] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d364d73a3d343b69e1b6595399dbc8c" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 36087 } } peers { permanent_uuid: "a5648ecf6ec94e41bfa844b04f324f75" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 46873 } } peers { permanent_uuid: "b58ec5338af94b31967bbd1eed5b38ee" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43489 } }
I20260811 05:14:18.409130 24619 leader_election.cc:288] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a5648ecf6ec94e41bfa844b04f324f75 (127.7.67.131:46873), b58ec5338af94b31967bbd1eed5b38ee (127.7.67.129:43489)
I20260811 05:14:18.413367 24368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:18.409396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:46873 (local address 127.7.67.130:53049)
0811 05:14:18.409549 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:18.409566 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:18.409667 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:18.409977 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:18.409980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:18.409998 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:14:18.410121 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:18.410126 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.410858 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.410863 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:18.412045 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:18.412056 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.412960 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.412963 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:18.413048 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:18.413069 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 05:14:18.413122 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 05:14:18.413211 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 05:14:18.413439 24611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:18.409499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:53049 (local address 127.7.67.131:46873)
0811 05:14:18.409631 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:18.409636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:18.409651 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:14:18.409809 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:18.409812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:18.409868 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:18.409949 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:14:18.410239 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.410744 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.412209 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.412833 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.412968 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:18.413035 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:18.413200 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 05:14:18.413274 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:14:18.413663 24622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:18.409399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:43489 (local address 127.7.67.130:36757)
0811 05:14:18.409911 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:18.409923 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:14:18.410053 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:14:18.410328 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:14:18.410331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:14:18.410347 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:14:18.410462 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:18.410468 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.411242 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.411246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:18.412316 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:14:18.412323 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.413225 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.413228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:14:18.413321 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:18.413342 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 05:14:18.413422 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 05:14:18.413491 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"mutex_wait_us":142,"thread_start_us":146,"threads_started":1}
I20260811 05:14:18.413713 24612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:18.409564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:36757 (local address 127.7.67.129:43489)
0811 05:14:18.409701 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:18.409708 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:18.410016 (+ 308us) server_negotiation.cc:407] Connection header received
0811 05:14:18.410172 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:18.410179 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:18.410225 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:18.410307 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:14:18.410577 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.411111 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.412450 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.413100 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.413235 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:18.413287 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:18.413459 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 05:14:18.413547 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 05:14:18.414178 24506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48c9078f0730415cabb86e763ee86e35" candidate_uuid: "4d364d73a3d343b69e1b6595399dbc8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5648ecf6ec94e41bfa844b04f324f75" is_pre_election: true
I20260811 05:14:18.414511 24506 raft_consensus.cc:2454] T 48c9078f0730415cabb86e763ee86e35 P a5648ecf6ec94e41bfa844b04f324f75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d364d73a3d343b69e1b6595399dbc8c in term 0.
I20260811 05:14:18.414578 24072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48c9078f0730415cabb86e763ee86e35" candidate_uuid: "4d364d73a3d343b69e1b6595399dbc8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b58ec5338af94b31967bbd1eed5b38ee" is_pre_election: true
I20260811 05:14:18.414921 24072 raft_consensus.cc:2454] T 48c9078f0730415cabb86e763ee86e35 P b58ec5338af94b31967bbd1eed5b38ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d364d73a3d343b69e1b6595399dbc8c in term 0.
I20260811 05:14:18.415083 24181 leader_election.cc:302] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d364d73a3d343b69e1b6595399dbc8c, a5648ecf6ec94e41bfa844b04f324f75; no voters:
I20260811 05:14:18.415385 24619 raft_consensus.cc:2781] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:14:18.415504 24619 raft_consensus.cc:482] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:14:18.415568 24619 raft_consensus.cc:3037] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:14:18.416728 24619 raft_consensus.cc:504] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d364d73a3d343b69e1b6595399dbc8c" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 36087 } } peers { permanent_uuid: "a5648ecf6ec94e41bfa844b04f324f75" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 46873 } } peers { permanent_uuid: "b58ec5338af94b31967bbd1eed5b38ee" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43489 } }
I20260811 05:14:18.417174 24619 leader_election.cc:288] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c [CANDIDATE]: Term 1 election: Requested vote from peers a5648ecf6ec94e41bfa844b04f324f75 (127.7.67.131:46873), b58ec5338af94b31967bbd1eed5b38ee (127.7.67.129:43489)
I20260811 05:14:18.417527 24506 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48c9078f0730415cabb86e763ee86e35" candidate_uuid: "4d364d73a3d343b69e1b6595399dbc8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5648ecf6ec94e41bfa844b04f324f75"
I20260811 05:14:18.417573 24072 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48c9078f0730415cabb86e763ee86e35" candidate_uuid: "4d364d73a3d343b69e1b6595399dbc8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b58ec5338af94b31967bbd1eed5b38ee"
I20260811 05:14:18.417672 24506 raft_consensus.cc:3037] T 48c9078f0730415cabb86e763ee86e35 P a5648ecf6ec94e41bfa844b04f324f75 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:14:18.417692 24072 raft_consensus.cc:3037] T 48c9078f0730415cabb86e763ee86e35 P b58ec5338af94b31967bbd1eed5b38ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:14:18.419088 24072 raft_consensus.cc:2454] T 48c9078f0730415cabb86e763ee86e35 P b58ec5338af94b31967bbd1eed5b38ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d364d73a3d343b69e1b6595399dbc8c in term 1.
I20260811 05:14:18.419088 24506 raft_consensus.cc:2454] T 48c9078f0730415cabb86e763ee86e35 P a5648ecf6ec94e41bfa844b04f324f75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d364d73a3d343b69e1b6595399dbc8c in term 1.
I20260811 05:14:18.419400 24183 leader_election.cc:302] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d364d73a3d343b69e1b6595399dbc8c, b58ec5338af94b31967bbd1eed5b38ee; no voters:
I20260811 05:14:18.419636 24619 raft_consensus.cc:2781] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:14:18.419901 24619 raft_consensus.cc:686] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c [term 1 LEADER]: Becoming Leader. State: Replica: 4d364d73a3d343b69e1b6595399dbc8c, State: Running, Role: LEADER
I20260811 05:14:18.420262 24619 consensus_queue.cc:237] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d364d73a3d343b69e1b6595399dbc8c" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 36087 } } peers { permanent_uuid: "a5648ecf6ec94e41bfa844b04f324f75" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 46873 } } peers { permanent_uuid: "b58ec5338af94b31967bbd1eed5b38ee" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43489 } }
I20260811 05:14:18.423681 23845 catalog_manager.cc:5495] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d364d73a3d343b69e1b6595399dbc8c (127.7.67.130). New cstate: current_term: 1 leader_uuid: "4d364d73a3d343b69e1b6595399dbc8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d364d73a3d343b69e1b6595399dbc8c" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 36087 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a5648ecf6ec94e41bfa844b04f324f75" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 46873 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b58ec5338af94b31967bbd1eed5b38ee" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43489 } health_report { overall_health: UNKNOWN } } }
I20260811 05:14:18.450003 24606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:18.446205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43002 (local address 127.7.67.130:36087)
0811 05:14:18.446369 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 05:14:18.446372 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:14:18.446386 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:14:18.446457 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:14:18.446459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:14:18.446513 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:14:18.446603 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:14:18.446933 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.447468 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.448420 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:14:18.448734 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:14:18.448840 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:14:18.449166 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:14:18.449610 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:14:18.449714 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 05:14:18.449813 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 05:14:18.449867 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 05:14:18.456188 24072 raft_consensus.cc:1261] T 48c9078f0730415cabb86e763ee86e35 P b58ec5338af94b31967bbd1eed5b38ee [term 1 FOLLOWER]: Refusing update from remote peer 4d364d73a3d343b69e1b6595399dbc8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:14:18.456188 24506 raft_consensus.cc:1261] T 48c9078f0730415cabb86e763ee86e35 P a5648ecf6ec94e41bfa844b04f324f75 [term 1 FOLLOWER]: Refusing update from remote peer 4d364d73a3d343b69e1b6595399dbc8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:14:18.457115 24623 consensus_queue.cc:1035] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b58ec5338af94b31967bbd1eed5b38ee" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 43489 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:14:18.457368 24619 consensus_queue.cc:1035] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5648ecf6ec94e41bfa844b04f324f75" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 46873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:14:18.465238 24629 mvcc.cc:205] Tried to move back new op lower bound from 7317197858628390912 to 7317197858492559360. Current Snapshot: MvccSnapshot[applied={T|T < 7317197858628390912}]
I20260811 05:14:18.466745 24630 mvcc.cc:205] Tried to move back new op lower bound from 7317197858628390912 to 7317197858492559360. Current Snapshot: MvccSnapshot[applied={T|T < 7317197858628390912}]
I20260811 05:14:18.474184 23845 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:14:18.474397 23845 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:14:18.477283 23845 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7975cf7d9bb146a6846758eea8ff03d3: Sending DeleteTablet for 3 replicas of tablet 48c9078f0730415cabb86e763ee86e35
I20260811 05:14:18.478260 24253 tablet_service.cc:1514] Processing DeleteTablet for tablet 48c9078f0730415cabb86e763ee86e35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:14:18 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:42988
I20260811 05:14:18.478259 24471 tablet_service.cc:1514] Processing DeleteTablet for tablet 48c9078f0730415cabb86e763ee86e35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:14:18 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:33988
I20260811 05:14:18.478310 24037 tablet_service.cc:1514] Processing DeleteTablet for tablet 48c9078f0730415cabb86e763ee86e35 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:14:18 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:55722
I20260811 05:14:18.478765 24644 tablet_replica.cc:330] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c: stopping tablet replica
I20260811 05:14:18.478792 24646 tablet_replica.cc:330] T 48c9078f0730415cabb86e763ee86e35 P a5648ecf6ec94e41bfa844b04f324f75: stopping tablet replica
I20260811 05:14:18.479064 24644 raft_consensus.cc:2229] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:14:18.479087 24646 raft_consensus.cc:2229] T 48c9078f0730415cabb86e763ee86e35 P a5648ecf6ec94e41bfa844b04f324f75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:14:18.479458 24646 raft_consensus.cc:2258] T 48c9078f0730415cabb86e763ee86e35 P a5648ecf6ec94e41bfa844b04f324f75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:14:18.479475 24644 raft_consensus.cc:2258] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:14:18.479656 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 23936
I20260811 05:14:18.489528 24644 ts_tablet_manager.cc:1905] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:14:18.493547 24644 ts_tablet_manager.cc:1918] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:14:18.493655 24644 log.cc:1196] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/48c9078f0730415cabb86e763ee86e35
I20260811 05:14:18.493988 24644 ts_tablet_manager.cc:1939] T 48c9078f0730415cabb86e763ee86e35 P 4d364d73a3d343b69e1b6595399dbc8c: Deleting consensus metadata
I20260811 05:14:18.494264 24646 ts_tablet_manager.cc:1905] T 48c9078f0730415cabb86e763ee86e35 P a5648ecf6ec94e41bfa844b04f324f75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:14:18.495599 23826 catalog_manager.cc:4841] TS 4d364d73a3d343b69e1b6595399dbc8c (127.7.67.130:36087): tablet 48c9078f0730415cabb86e763ee86e35 (table test-table [id=ef408db45cd44d9984163a777077d4c6]) successfully deleted
I20260811 05:14:18.497259 24646 ts_tablet_manager.cc:1918] T 48c9078f0730415cabb86e763ee86e35 P a5648ecf6ec94e41bfa844b04f324f75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:14:18.497356 24646 log.cc:1196] T 48c9078f0730415cabb86e763ee86e35 P a5648ecf6ec94e41bfa844b04f324f75: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/48c9078f0730415cabb86e763ee86e35
I20260811 05:14:18.497705 24646 ts_tablet_manager.cc:1939] T 48c9078f0730415cabb86e763ee86e35 P a5648ecf6ec94e41bfa844b04f324f75: Deleting consensus metadata
I20260811 05:14:18.499560 23827 catalog_manager.cc:4841] TS a5648ecf6ec94e41bfa844b04f324f75 (127.7.67.131:46873): tablet 48c9078f0730415cabb86e763ee86e35 (table test-table [id=ef408db45cd44d9984163a777077d4c6]) successfully deleted
W20260811 05:14:18.506661 23828 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.129:43489 (error 108)
I20260811 05:14:18.506897 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 24157
I20260811 05:14:18.513537 24602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:14:18.513088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:43489 (local address 127.0.0.1:55736)
0811 05:14:18.513295 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 05:14:18.513409 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.129:43489: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 05:14:18.513790 23828 catalog_manager.cc:4584] TS b58ec5338af94b31967bbd1eed5b38ee (127.7.67.129:43489): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 48c9078f0730415cabb86e763ee86e35: Network error: Client connection negotiation failed: client connection to 127.7.67.129:43489: connect: Connection refused (error 111)
I20260811 05:14:18.521757 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 24373
I20260811 05:14:18.539311 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 23799
2026-08-11T05:14:18Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2622 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:14:18 dist-test-slave-04bb krb5kdc[24653](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:14:18 dist-test-slave-04bb krb5kdc[24653](info): set up 2 sockets
Aug 11 05:14:18 dist-test-slave-04bb krb5kdc[24653](info): commencing operation
krb5kdc: starting...
W20260811 05:14:19.823868 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:14:19 dist-test-slave-04bb krb5kdc[24653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:14:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:14:19Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.67.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.67.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.67.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/rangeradmin_127.7.67.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.67.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/rangeradmin_127.7.67.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.67.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.67.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.67.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/rangerlookup_127.7.67.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.67.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/rangerlookup_127.7.67.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.67.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.67.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.67.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/HTTP_127.7.67.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.67.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/HTTP_127.7.67.148@KRBTEST.COM.keytab.
I20260811 05:14:20.015486 7438 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasktf25Hl/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/postgres -l logfile start
2026-08-11 05:14:22.663 UTC [24703] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 05:14:22.663 UTC [24703] LOG: listening on IPv4 address "127.7.67.148", port 39785
2026-08-11 05:14:22.667 UTC [24703] LOG: listening on Unix socket "/tmp/.s.PGSQL.39785"
2026-08-11 05:14:22.689 UTC [24708] LOG: database system was shut down at 2026-08-11 05:14:21 UTC
2026-08-11 05:14:22.693 UTC [24703] LOG: database system is ready to accept connections
I20260811 05:14:23.858635 7438 mini_postgres.cc:96] Postgres bound to 39785
2026-08-11 05:14:23.866 UTC [24720] FATAL: database "slave" does not exist
127.7.67.148:39785 - accepting connections
I20260811 05:14:23.867244 7438 mini_ranger.cc:161] Starting Ranger...
I20260811 05:14:23.883944 7438 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 05:14:24.393193 7438 mini_ranger.cc:87] Created ranger Postgres database
I20260811 05:14:24.393327 7438 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ranger-admin
2026-08-11 05:14:24,674 [I] DB FLAVOR :POSTGRES
2026-08-11 05:14:24,675 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 05:14:24,675 [I] Checking connection..
2026-08-11 05:14:24,675 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:39785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 05:14:24,928 [I] Checking connection passed.
2026-08-11 05:14:24,928 [I] --------- Verifying version history table ---------
2026-08-11 05:14:24,928 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:39785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 05:14:25,194 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 05:14:25,195 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:39785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 05:14:25,454 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 05:14:25,455 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 05:14:25,455 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:39785/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 05:14:25.683 UTC [24830] WARNING: there is no transaction in progress
2026-08-11 05:14:25,692 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 05:14:25,692 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:39785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 05:14:25,959 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 05:14:25,960 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 05:14:25,960 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:39785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 05:14:26,213 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:39785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 05:14:26,452 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:39785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-04bb.c.gcp-upstream.internal','N') ;"
2026-08-11 05:14:26,688 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:39785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 05:14:26,937 [I] Table x_portal_user does not exist in database ranger
2026-08-11 05:14:26,937 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:39785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 05:14:27,202 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 05:14:27,202 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:39785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 05:14:27,479 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:39785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 05:14:27,714 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 05:14:27,714 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:39785/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 05:14:28.745 UTC [25008] WARNING: there is no transaction in progress
2026-08-11 05:14:28.762 UTC [25008] WARNING: there is no transaction in progress
2026-08-11 05:14:28.781 UTC [25008] WARNING: there is no transaction in progress
2026-08-11 05:14:28.796 UTC [25008] WARNING: there is no transaction in progress
2026-08-11 05:14:28.813 UTC [25008] WARNING: there is no transaction in progress
2026-08-11 05:14:28.831 UTC [25008] WARNING: there is no transaction in progress
2026-08-11 05:14:28.890 UTC [25008] WARNING: there is no transaction in progress
2026-08-11 05:14:28.898 UTC [25008] WARNING: there is no transaction in progress
2026-08-11 05:14:28.913 UTC [25008] WARNING: there is no transaction in progress
2026-08-11 05:14:28.924 UTC [25008] WARNING: there is no transaction in progress
2026-08-11 05:14:28.935 UTC [25008] WARNING: there is no transaction in progress
2026-08-11 05:14:28.949 UTC [25008] WARNING: there is no transaction in progress
2026-08-11 05:14:28.959 UTC [25008] WARNING: there is no transaction in progress
2026-08-11 05:14:28.966 UTC [25008] WARNING: there is no transaction in progress
2026-08-11 05:14:28.974 UTC [25008] WARNING: there is no transaction in progress
2026-08-11 05:14:28.981 UTC [25008] WARNING: there is no transaction in progress
2026-08-11 05:14:29,688 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 05:14:29,689 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:39785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 05:14:29,935 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 05:14:29,935 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:39785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 05:14:30,208 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 05:14:30,208 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:39785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 05:14:30,479 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 05:14:30,479 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:39785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 05:14:30,725 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:39785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 05:14:31,003 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:39785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-04bb.c.gcp-upstream.internal';"
2026-08-11 05:14:31,263 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 05:14:31,263 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:39785/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 05:14:31,536 [I] DB_PATCHES have already been applied
I20260811 05:14:31.543045 7438 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ranger-admin:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasktf25Hl/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 05:14:31.543118 7438 mini_ranger.cc:193] Using host: 127.7.67.148
I20260811 05:14:31.547906 7438 mini_ranger.cc:227] Ranger admin URL: http://127.7.67.148:46503
Aug 11, 2026 5:14:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 5:14:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 5:14:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 5:14:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 5:14:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 5:14:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 5:14:32 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 05:14:32.953068 7438 mini_ranger.cc:160] Time spent starting Ranger: real 9.086s user 0.000s sys 0.013s
Aug 11, 2026 5:14:53 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 5:14:53 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 11, 2026 5:14:53 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 5:14:53 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 5:14:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 5:14:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 5:14:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 5:14:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 5:14:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 5:14:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 5:14:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 5:14:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 5:14:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 5:14:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 5:14:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 5:14:53 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 5:14:53 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 5:14:54 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 05:14:55.253428 7438 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.67.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.67.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.67.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/rangerkms_127.7.67.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.67.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/rangerkms_127.7.67.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.67.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.67.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.67.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/HTTP_127.7.67.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.67.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/HTTP_127.7.67.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 05:14:55 dist-test-slave-04bb krb5kdc[24653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425295, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 05:14:55.345767 7438 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 05:14:55.356549 7438 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 05:14:56.305092 7438 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 05:14:56.305197 7438 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ranger-kms
I20260811 05:14:56.305318 7438 mini_ranger_kms.cc:227] Using postgres at 127.7.67.148:39785
2026-08-11 05:14:56,610 [I] DB FLAVOR :POSTGRES
2026-08-11 05:14:56,610 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 05:14:56,610 [I] Checking connection
2026-08-11 05:14:56,611 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:39785/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 05:14:56,855 [I] connection success
2026-08-11 05:14:56,856 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 05:14:56,856 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 05:14:56,856 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:39785/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 05:14:57,111 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 05:14:57,111 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 05:14:57,112 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 05:14:57,112 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:39785/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 05:14:57,383 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 05:14:57.878479 7438 mini_ranger_kms.cc:310] Created kms service
I20260811 05:14:58.094427 7438 mini_ranger_kms.cc:326] Created kudu user
I20260811 05:14:58.139282 7438 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 05:14:58 dist-test-slave-04bb krb5kdc[24653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425295, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.67.148@KRBTEST.COM
I20260811 05:14:58.603255 7438 mini_ranger_kms.cc:384] Added ranger policy
I20260811 05:14:58.603504 7438 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ranger-kms:/tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 05:14:58.603547 7438 mini_ranger_kms.cc:240] Using host: 127.7.67.148
I20260811 05:14:58.610613 7438 mini_ranger_kms.cc:276] Ranger KMS PID: 25305
I20260811 05:14:58.611124 7438 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.67.148:53013
Aug 11, 2026 5:14:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 5:14:59 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 5:15:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 5:15:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 5:15:00 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 5:15:00 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260811 05:15:01.483422 7438 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.138s user 0.006s sys 0.287s
I20260811 05:15:01.483579 7438 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 05:15:01.483745 7438 mini_ranger_kms.cc:401] 127.7.67.148:53013/kms/v1/keys
Aug 11 05:15:01 dist-test-slave-04bb krb5kdc[24653](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786425301, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.67.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 05:15:01 dist-test-slave-04bb krb5kdc[24653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425301, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.67.148@KRBTEST.COM for HTTP/127.7.67.148@KRBTEST.COM
Aug 11 05:15:02 dist-test-slave-04bb krb5kdc[24653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425301, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.67.148@KRBTEST.COM for HTTP/127.7.67.148@KRBTEST.COM
Aug 11 05:15:04 dist-test-slave-04bb krb5kdc[24653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:15:04.926198 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:41155
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:44935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:41155
--ranger_config_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.67.148:53013/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:15:05.100455 25384 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:05.100788 25384 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:05.100831 25384 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:05.105120 25384 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:15:05.105185 25384 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:05.105206 25384 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:05.105222 25384 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:15:05.105237 25384 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:15:05.110890 25384 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:44935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.67.148:53013/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:41155
--ranger_config_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:41155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25384
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:15:05.111881 25384 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:05.112941 25384 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:05.122670 25393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:05.122702 25396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:05.122720 25394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:05.122819 25384 server_base.cc:1034] running on GCE node
I20260811 05:15:05.123592 25384 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:05.124910 25384 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:05.126116 25384 hybrid_clock.cc:648] HybridClock initialized: now 1786425305126104 us; error 41 us; skew 500 ppm
Aug 11 05:15:05 dist-test-slave-04bb krb5kdc[24653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425305, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:05.132433 25384 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:15:05.133417 25384 webserver.cc:460] Webserver started at http://127.7.67.190:45383/ using document root <none> and password file <none>
I20260811 05:15:05.134114 25384 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:05.134208 25384 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:05.134435 25384 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 05:15:05 dist-test-slave-04bb krb5kdc[24653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425305, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for HTTP/127.7.67.148@KRBTEST.COM
I20260811 05:15:05.195600 25384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "eee3c8c411a04a6da7d6f0e275388df6"
format_stamp: "Formatted at 2026-08-11 05:15:05 on dist-test-slave-04bb"
server_key: "4465a389073a85c5c4c8c4c0409f6264"
server_key_iv: "c53e123d527217cbfe3a713584f36463"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:05.196424 25384 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "eee3c8c411a04a6da7d6f0e275388df6"
format_stamp: "Formatted at 2026-08-11 05:15:05 on dist-test-slave-04bb"
server_key: "4465a389073a85c5c4c8c4c0409f6264"
server_key_iv: "c53e123d527217cbfe3a713584f36463"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:05.201993 25384 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 05:15:05.222105 25405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:05.223918 25384 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:15:05.224066 25384 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "eee3c8c411a04a6da7d6f0e275388df6"
format_stamp: "Formatted at 2026-08-11 05:15:05 on dist-test-slave-04bb"
server_key: "4465a389073a85c5c4c8c4c0409f6264"
server_key_iv: "c53e123d527217cbfe3a713584f36463"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:05.224217 25384 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:05.249033 25384 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:05.252692 25384 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:05.252933 25384 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:05.266861 25384 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:41155
I20260811 05:15:05.267087 25497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:41155 every 8 connection(s)
I20260811 05:15:05.268147 25384 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:15:05.271282 25498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:15:05.277662 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 25384
I20260811 05:15:05.277658 25498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eee3c8c411a04a6da7d6f0e275388df6: Bootstrap starting.
I20260811 05:15:05.277809 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:15:05.278095 7438 external_mini_cluster.cc:1428] Setting key 6e4f89a32d10afefeee2eeea6ab5484e
I20260811 05:15:05.282300 25498 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eee3c8c411a04a6da7d6f0e275388df6: Neither blocks nor log segments found. Creating new log.
I20260811 05:15:05.283630 25498 log.cc:824] T 00000000000000000000000000000000 P eee3c8c411a04a6da7d6f0e275388df6: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:05.285954 25498 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eee3c8c411a04a6da7d6f0e275388df6: No bootstrap required, opened a new log
Aug 11 05:15:05 dist-test-slave-04bb krb5kdc[24653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:15:05.291821 25498 raft_consensus.cc:348] T 00000000000000000000000000000000 P eee3c8c411a04a6da7d6f0e275388df6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eee3c8c411a04a6da7d6f0e275388df6" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 41155 } }
I20260811 05:15:05.292086 25498 raft_consensus.cc:374] T 00000000000000000000000000000000 P eee3c8c411a04a6da7d6f0e275388df6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:15:05.292176 25498 raft_consensus.cc:729] T 00000000000000000000000000000000 P eee3c8c411a04a6da7d6f0e275388df6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eee3c8c411a04a6da7d6f0e275388df6, State: Initialized, Role: FOLLOWER
I20260811 05:15:05.292757 25498 consensus_queue.cc:260] T 00000000000000000000000000000000 P eee3c8c411a04a6da7d6f0e275388df6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eee3c8c411a04a6da7d6f0e275388df6" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 41155 } }
I20260811 05:15:05.292932 25498 raft_consensus.cc:388] T 00000000000000000000000000000000 P eee3c8c411a04a6da7d6f0e275388df6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:15:05.293020 25498 raft_consensus.cc:482] T 00000000000000000000000000000000 P eee3c8c411a04a6da7d6f0e275388df6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:15:05.293159 25498 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eee3c8c411a04a6da7d6f0e275388df6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:15:05.294533 25498 raft_consensus.cc:504] T 00000000000000000000000000000000 P eee3c8c411a04a6da7d6f0e275388df6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eee3c8c411a04a6da7d6f0e275388df6" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 41155 } }
I20260811 05:15:05.294919 25498 leader_election.cc:302] T 00000000000000000000000000000000 P eee3c8c411a04a6da7d6f0e275388df6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eee3c8c411a04a6da7d6f0e275388df6; no voters:
I20260811 05:15:05.295346 25498 leader_election.cc:288] T 00000000000000000000000000000000 P eee3c8c411a04a6da7d6f0e275388df6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:15:05.295825 25505 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eee3c8c411a04a6da7d6f0e275388df6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:15:05.296192 25505 raft_consensus.cc:686] T 00000000000000000000000000000000 P eee3c8c411a04a6da7d6f0e275388df6 [term 1 LEADER]: Becoming Leader. State: Replica: eee3c8c411a04a6da7d6f0e275388df6, State: Running, Role: LEADER
I20260811 05:15:05.296653 25505 consensus_queue.cc:237] T 00000000000000000000000000000000 P eee3c8c411a04a6da7d6f0e275388df6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eee3c8c411a04a6da7d6f0e275388df6" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 41155 } }
I20260811 05:15:05.297574 25501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:05.279573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37272 (local address 127.7.67.190:41155)
0811 05:15:05.280134 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:05.280192 (+ 58us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:05.280268 (+ 76us) server_negotiation.cc:407] Connection header received
0811 05:15:05.281436 (+ 1168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:05.281463 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:05.281898 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:05.282300 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:05.283164 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:05.284652 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:05.285935 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:05.286788 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:05.286952 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:05.290892 (+ 3940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:05.290901 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:05.290957 (+ 56us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:05.290994 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:05.293492 (+ 2498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:05.294036 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:05.294043 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:05.294049 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:05.294200 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:05.294567 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:05.294571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:05.294572 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:05.295139 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:05.295340 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:05.296181 (+ 841us) server_negotiation.cc:299] Negotiation successful
0811 05:15:05.296475 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":172,"threads_started":1}
I20260811 05:15:05.298148 25505 sys_catalog.cc:455] T 00000000000000000000000000000000 P eee3c8c411a04a6da7d6f0e275388df6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eee3c8c411a04a6da7d6f0e275388df6. Latest consensus state: current_term: 1 leader_uuid: "eee3c8c411a04a6da7d6f0e275388df6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eee3c8c411a04a6da7d6f0e275388df6" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 41155 } } }
I20260811 05:15:05.298285 25505 sys_catalog.cc:458] T 00000000000000000000000000000000 P eee3c8c411a04a6da7d6f0e275388df6 [sys.catalog]: This master's current role is: LEADER
I20260811 05:15:05.296811 25498 sys_catalog.cc:564] T 00000000000000000000000000000000 P eee3c8c411a04a6da7d6f0e275388df6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:15:05.299178 25516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:15:05.301308 25504 sys_catalog.cc:455] T 00000000000000000000000000000000 P eee3c8c411a04a6da7d6f0e275388df6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eee3c8c411a04a6da7d6f0e275388df6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eee3c8c411a04a6da7d6f0e275388df6" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 41155 } } }
I20260811 05:15:05.301414 25504 sys_catalog.cc:458] T 00000000000000000000000000000000 P eee3c8c411a04a6da7d6f0e275388df6 [sys.catalog]: This master's current role is: LEADER
I20260811 05:15:05.303431 25516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:15:05.304888 25498 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 05:15:05.323143 25516 catalog_manager.cc:1347] Generated new cluster ID: 62dddc6c48e84aacaa3506edaece7971
I20260811 05:15:05.323243 25516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:15:05.335147 25516 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:15:05.336606 25516 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:15:05.348043 25516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eee3c8c411a04a6da7d6f0e275388df6: Generated new TSK 0
I20260811 05:15:05.349074 25516 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
I20260811 05:15:06.273527 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.67.148:53013/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:41155
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:44935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:15:06.434669 25565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:06.434979 25565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:06.435081 25565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:06.439340 25565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:06.439504 25565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:06.439671 25565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:15:06.445833 25565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:44935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.67.148:53013/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:41155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:15:06.447175 25565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:06.448310 25565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:06.460309 25577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:06.460403 25578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:06.460515 25580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:06.461181 25565 server_base.cc:1034] running on GCE node
I20260811 05:15:06.461654 25565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:06.462373 25565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:06.463558 25565 hybrid_clock.cc:648] HybridClock initialized: now 1786425306463545 us; error 31 us; skew 500 ppm
Aug 11 05:15:06 dist-test-slave-04bb krb5kdc[24653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425306, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:06.467223 25565 init.cc:377] Logged in from keytab as kudu/127.7.67.129@KRBTEST.COM (short username kudu)
I20260811 05:15:06.468374 25565 webserver.cc:460] Webserver started at http://127.7.67.129:37953/ using document root <none> and password file <none>
I20260811 05:15:06.469020 25565 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:06.469091 25565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:06.469358 25565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 05:15:06 dist-test-slave-04bb krb5kdc[24653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425306, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for HTTP/127.7.67.148@KRBTEST.COM
I20260811 05:15:06.487668 25565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "b9127bd978bd46dfbd2a747ec3eb988d"
format_stamp: "Formatted at 2026-08-11 05:15:06 on dist-test-slave-04bb"
server_key: "8088b119374b48da71f1d81870251ce5"
server_key_iv: "621bdaf27242b9b1dba52847fc5eb077"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:06.488649 25565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "b9127bd978bd46dfbd2a747ec3eb988d"
format_stamp: "Formatted at 2026-08-11 05:15:06 on dist-test-slave-04bb"
server_key: "8088b119374b48da71f1d81870251ce5"
server_key_iv: "621bdaf27242b9b1dba52847fc5eb077"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:06.497617 25565 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.003s sys 0.001s
I20260811 05:15:06.523734 25596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:06.525815 25565 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 05:15:06.525992 25565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "b9127bd978bd46dfbd2a747ec3eb988d"
format_stamp: "Formatted at 2026-08-11 05:15:06 on dist-test-slave-04bb"
server_key: "8088b119374b48da71f1d81870251ce5"
server_key_iv: "621bdaf27242b9b1dba52847fc5eb077"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:06.526201 25565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:06.547484 25565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:06.551877 25565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:06.552104 25565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:06.553033 25565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:06.554435 25565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:15:06.554553 25565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260811 05:15:06.554734 25565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:15:06.554864 25565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:06.575910 25565 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:44663
I20260811 05:15:06.578662 25565 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:15:06.580272 25790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:44663 every 8 connection(s)
I20260811 05:15:06.583027 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 25565
I20260811 05:15:06.583171 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:15:06.583518 7438 external_mini_cluster.cc:1428] Setting key aaa29b331d6162f05bdbf2325a0f36cf
Aug 11 05:15:06 dist-test-slave-04bb krb5kdc[24653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425306, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:15:06.601877 25801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:06.584338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:40981 (local address 127.7.67.190:41155)
0811 05:15:06.586662 (+ 2324us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:06.586667 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:06.586681 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:15:06.586722 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:06.586725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:06.586771 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:06.586860 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:06.588717 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:06.589621 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:06.591445 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:06.591846 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:06.591929 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:06.596725 (+ 4796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:06.596729 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:06.596732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:06.596761 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:06.598902 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:06.599567 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:06.599571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:06.599574 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:06.599688 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:06.600264 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:06.600268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:06.600270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:06.600576 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:06.600679 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:06.601491 (+ 812us) server_negotiation.cc:299] Negotiation successful
0811 05:15:06.601665 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2243,"thread_start_us":120,"threads_started":1}
I20260811 05:15:06.603932 25797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:06.583032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:41155 (local address 127.7.67.129:40981)
0811 05:15:06.584927 (+ 1895us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:06.584991 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:06.586222 (+ 1231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:06.587180 (+ 958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:06.587189 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:06.587697 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:06.588514 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:06.588559 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:06.589786 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:06.589789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:06.591286 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:06.591310 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:06.592148 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:06.592152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:06.592608 (+ 456us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:06.594412 (+ 1804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:06.594434 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:06.596481 (+ 2047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:06.599049 (+ 2568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:06.599056 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:06.599073 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:06.599411 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:06.599896 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:06.599900 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:06.599902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:06.600103 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:06.600759 (+ 656us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:06.600767 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:06.601149 (+ 382us) client_negotiation.cc:769] Sending connection context
0811 05:15:06.602077 (+ 928us) client_negotiation.cc:240] Negotiation successful
0811 05:15:06.602558 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1608,"thread_start_us":224,"threads_started":1}
I20260811 05:15:06.605578 25791 heartbeater.cc:347] Connected to a master server at 127.7.67.190:41155
I20260811 05:15:06.605846 25791 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:06.606418 25791 heartbeater.cc:511] Master 127.7.67.190:41155 requested a full tablet report, sending...
I20260811 05:15:06.608255 25436 ts_manager.cc:194] Registered new tserver with Master: b9127bd978bd46dfbd2a747ec3eb988d (127.7.67.129:44663)
I20260811 05:15:06.609874 25436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.129@KRBTEST.COM'} at 127.7.67.129:40981
WARNING: no policy specified for kudu/127.7.67.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
I20260811 05:15:06.640834 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.67.148:53013/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:41155
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:44935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:15:06 dist-test-slave-04bb krb5kdc[24653](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786425306, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 05:15:06.851059 25807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:06.851517 25807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:06.851601 25807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:06.858140 25807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:06.858235 25807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:06.858323 25807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:15:06.866397 25807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:44935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.67.148:53013/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:41155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:15:06.867892 25807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:06.869432 25807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:06.881562 25821 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:06.881534 25817 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:06.881711 25807 server_base.cc:1034] running on GCE node
W20260811 05:15:06.882021 25819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:06.882452 25807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:06.883119 25807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:06.884326 25807 hybrid_clock.cc:648] HybridClock initialized: now 1786425306884292 us; error 54 us; skew 500 ppm
Aug 11 05:15:06 dist-test-slave-04bb krb5kdc[24653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425306, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:06.888657 25807 init.cc:377] Logged in from keytab as kudu/127.7.67.130@KRBTEST.COM (short username kudu)
I20260811 05:15:06.889868 25807 webserver.cc:460] Webserver started at http://127.7.67.130:35303/ using document root <none> and password file <none>
I20260811 05:15:06.890659 25807 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:06.890764 25807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:06.891094 25807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 05:15:06 dist-test-slave-04bb krb5kdc[24653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425306, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for HTTP/127.7.67.148@KRBTEST.COM
I20260811 05:15:06.912608 25807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "220d4a0f3c884451ab1ab5c3629b22d1"
format_stamp: "Formatted at 2026-08-11 05:15:06 on dist-test-slave-04bb"
server_key: "a9e0aad03d3bd36b98426ab25095d5e3"
server_key_iv: "bcfa631042af5372166471e13e4c789a"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:06.913472 25807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "220d4a0f3c884451ab1ab5c3629b22d1"
format_stamp: "Formatted at 2026-08-11 05:15:06 on dist-test-slave-04bb"
server_key: "a9e0aad03d3bd36b98426ab25095d5e3"
server_key_iv: "bcfa631042af5372166471e13e4c789a"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:06.931064 25807 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.001s sys 0.004s
I20260811 05:15:06.962873 25834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:06.964812 25807 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 05:15:06.964946 25807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "220d4a0f3c884451ab1ab5c3629b22d1"
format_stamp: "Formatted at 2026-08-11 05:15:06 on dist-test-slave-04bb"
server_key: "a9e0aad03d3bd36b98426ab25095d5e3"
server_key_iv: "bcfa631042af5372166471e13e4c789a"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:06.965087 25807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:06.991778 25807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:06.996058 25807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:06.996336 25807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:06.997184 25807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:06.998358 25807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:15:06.998464 25807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:06.998564 25807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:15:06.998625 25807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:07.010558 25807 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:39407
I20260811 05:15:07.011816 25807 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:15:07.015961 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 25807
I20260811 05:15:07.016094 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:15:07.016409 7438 external_mini_cluster.cc:1428] Setting key 83ca80fa1711f941b26840987abfffc9
I20260811 05:15:07.021157 26019 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:39407 every 8 connection(s)
Aug 11 05:15:07 dist-test-slave-04bb krb5kdc[24653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425306, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:15:07.038605 25801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:07.020867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:39721 (local address 127.7.67.190:41155)
0811 05:15:07.020996 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:07.021002 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:07.022499 (+ 1497us) server_negotiation.cc:407] Connection header received
0811 05:15:07.023702 (+ 1203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:07.023706 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:07.023762 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:07.023817 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:07.025551 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.026386 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.028125 (+ 1739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.028494 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.028611 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:07.033676 (+ 5065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:07.033705 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:07.033708 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:07.033739 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:07.035848 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:07.036576 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:07.036580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:07.036581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:07.036679 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:07.037133 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:07.037138 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:07.037140 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:07.037401 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:07.037497 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:07.038266 (+ 769us) server_negotiation.cc:299] Negotiation successful
0811 05:15:07.038415 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 05:15:07.040596 26024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:07.019638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:41155 (local address 127.7.67.130:39721)
0811 05:15:07.022348 (+ 2710us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:07.022404 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:07.023450 (+ 1046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:07.024228 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:07.024237 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:07.024640 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:07.025349 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:07.025372 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.026552 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.026555 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:07.027982 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:07.027994 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.028823 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.028826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:07.029091 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:07.030244 (+ 1153us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:07.030279 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:07.033466 (+ 3187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:07.035996 (+ 2530us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:07.036004 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:07.036022 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:07.036420 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:07.036807 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:07.036810 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:07.036813 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:07.037006 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:07.037576 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:07.037583 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:07.037938 (+ 355us) client_negotiation.cc:769] Sending connection context
0811 05:15:07.039112 (+ 1174us) client_negotiation.cc:240] Negotiation successful
0811 05:15:07.039501 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2469,"thread_start_us":152,"threads_started":1}
I20260811 05:15:07.042395 26021 heartbeater.cc:347] Connected to a master server at 127.7.67.190:41155
WARNING: no policy specified for kudu/127.7.67.131@KRBTEST.COM; defaulting to no policy
I20260811 05:15:07.042663 26021 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:07.043313 26021 heartbeater.cc:511] Master 127.7.67.190:41155 requested a full tablet report, sending...
I20260811 05:15:07.044629 25436 ts_manager.cc:194] Registered new tserver with Master: 220d4a0f3c884451ab1ab5c3629b22d1 (127.7.67.130:39407)
I20260811 05:15:07.045423 25436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.130@KRBTEST.COM'} at 127.7.67.130:39721
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
I20260811 05:15:07.072386 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.67.148:53013/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:41155
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:44935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:15:07.303056 26029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:07.303639 26029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:07.303798 26029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:07.309993 26029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:07.310134 26029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:07.310281 26029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:15:07.318446 26029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:44935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.67.148:53013/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:41155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:15:07.320186 26029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:07.321698 26029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:07.338011 26036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:07.338606 26035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:07.339862 26029 server_base.cc:1034] running on GCE node
W20260811 05:15:07.340349 26039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:07.342639 26029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:07.343514 26029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:07.344897 26029 hybrid_clock.cc:648] HybridClock initialized: now 1786425307344819 us; error 95 us; skew 500 ppm
Aug 11 05:15:07 dist-test-slave-04bb krb5kdc[24653](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425307, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:07.349742 26029 init.cc:377] Logged in from keytab as kudu/127.7.67.131@KRBTEST.COM (short username kudu)
I20260811 05:15:07.350984 26029 webserver.cc:460] Webserver started at http://127.7.67.131:38337/ using document root <none> and password file <none>
I20260811 05:15:07.351819 26029 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:07.351980 26029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:07.352301 26029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 05:15:07 dist-test-slave-04bb krb5kdc[24653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425307, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for HTTP/127.7.67.148@KRBTEST.COM
I20260811 05:15:07.391047 26029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "ebeb41dfb7cc44c7a54e273e837de963"
format_stamp: "Formatted at 2026-08-11 05:15:07 on dist-test-slave-04bb"
server_key: "53d5abb40df359bdd708fde1b9d3a60a"
server_key_iv: "27eca62d17ca14c010b66fc49a24b559"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:07.392095 26029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "ebeb41dfb7cc44c7a54e273e837de963"
format_stamp: "Formatted at 2026-08-11 05:15:07 on dist-test-slave-04bb"
server_key: "53d5abb40df359bdd708fde1b9d3a60a"
server_key_iv: "27eca62d17ca14c010b66fc49a24b559"
server_key_version: "kuduclusterkey@0"
Aug 11 05:15:07 dist-test-slave-04bb krb5kdc[24653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425306, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for HTTP/127.7.67.148@KRBTEST.COM
I20260811 05:15:07.410336 26029 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.002s sys 0.002s
I20260811 05:15:07.450039 26052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:07.461474 26029 fs_manager.cc:725] Time spent opening block manager: real 0.018s user 0.002s sys 0.002s
I20260811 05:15:07.461661 26029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "ebeb41dfb7cc44c7a54e273e837de963"
format_stamp: "Formatted at 2026-08-11 05:15:07 on dist-test-slave-04bb"
server_key: "53d5abb40df359bdd708fde1b9d3a60a"
server_key_iv: "27eca62d17ca14c010b66fc49a24b559"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:07.461823 26029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:07.513973 26029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:07.517508 26029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:07.517805 26029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:07.518713 26029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:07.519960 26029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:15:07.520079 26029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:07.520205 26029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:15:07.520278 26029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:07.556981 26029 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:44185
I20260811 05:15:07.559233 26232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:44185 every 8 connection(s)
I20260811 05:15:07.559737 26029 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:15:07.569526 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 26029
I20260811 05:15:07.569671 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:15:07.570017 7438 external_mini_cluster.cc:1428] Setting key 79ff819e27d97397fd22d7cb93f98c20
Aug 11 05:15:07 dist-test-slave-04bb krb5kdc[24653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425307, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:15:07.580996 26236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:07.562424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:47267 (local address 127.7.67.190:41155)
0811 05:15:07.562850 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:07.562855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:07.564646 (+ 1791us) server_negotiation.cc:407] Connection header received
0811 05:15:07.566056 (+ 1410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:07.566060 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:07.566121 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:07.566238 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:07.568099 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.569100 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.571230 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.571560 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.571684 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:07.575720 (+ 4036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:07.575725 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:07.575728 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:07.575758 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:07.577762 (+ 2004us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:07.578576 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:07.578580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:07.578582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:07.578678 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:07.579201 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:07.579205 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:07.579207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:07.579447 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:07.579555 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:07.580680 (+ 1125us) server_negotiation.cc:299] Negotiation successful
0811 05:15:07.580845 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":108,"threads_started":1}
I20260811 05:15:07.582527 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:07.563028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:41155 (local address 127.7.67.131:47267)
0811 05:15:07.564418 (+ 1390us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:07.564482 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:07.565775 (+ 1293us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:07.566477 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:07.566487 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:07.566992 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:07.567889 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:07.567920 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.569281 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.569284 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:07.571004 (+ 1720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:07.571016 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.571906 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.571910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:07.572200 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:07.573309 (+ 1109us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:07.573331 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:07.575490 (+ 2159us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:07.577970 (+ 2480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:07.577978 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:07.577999 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:07.578443 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:07.578810 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:07.578814 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:07.578816 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:07.579039 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:07.579599 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:07.579626 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:07.580274 (+ 648us) client_negotiation.cc:769] Sending connection context
0811 05:15:07.581249 (+ 975us) client_negotiation.cc:240] Negotiation successful
0811 05:15:07.581610 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1133,"thread_start_us":211,"threads_started":1}
I20260811 05:15:07.584187 26233 heartbeater.cc:347] Connected to a master server at 127.7.67.190:41155
I20260811 05:15:07.584470 26233 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:07.585222 26233 heartbeater.cc:511] Master 127.7.67.190:41155 requested a full tablet report, sending...
I20260811 05:15:07.586400 25436 ts_manager.cc:194] Registered new tserver with Master: ebeb41dfb7cc44c7a54e273e837de963 (127.7.67.131:44185)
I20260811 05:15:07.587069 25436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.131@KRBTEST.COM'} at 127.7.67.131:47267
I20260811 05:15:07.598052 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:15:07.610677 26236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:07.600488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36630 (local address 127.7.67.190:41155)
0811 05:15:07.600710 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:07.600727 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:07.600902 (+ 175us) server_negotiation.cc:407] Connection header received
0811 05:15:07.600991 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:07.600994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:07.601082 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:07.601199 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:07.601973 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.603454 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.604620 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.605274 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.605482 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:07.606992 (+ 1510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:07.606996 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:07.606999 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:07.607026 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:07.608793 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:07.609297 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:07.609301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:07.609303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:07.609399 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:07.609714 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:07.609718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:07.609720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:07.609938 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:07.610096 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:07.610357 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 05:15:07.610493 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 05:15:07.612840 25791 heartbeater.cc:503] Master 127.7.67.190:41155 was elected leader, sending a full tablet report...
I20260811 05:15:07.616604 25435 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36630:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:15:07.620129 25435 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:15:07.639735 26254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:07.630179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:44663 (local address 127.0.0.1:33166)
0811 05:15:07.632886 (+ 2707us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:07.632936 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:07.633067 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:07.634172 (+ 1105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:07.634178 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:07.634201 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:07.634361 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:07.634372 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.636054 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.636060 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:07.637614 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:07.637626 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.639028 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.639032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:07.639206 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:07.639259 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 05:15:07.639372 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 05:15:07.639525 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2472,"mutex_wait_us":4,"thread_start_us":960,"threads_started":1}
I20260811 05:15:07.640444 26253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:07.630647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33166 (local address 127.7.67.129:44663)
0811 05:15:07.633669 (+ 3022us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:07.633674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:07.633694 (+ 20us) server_negotiation.cc:407] Connection header received
0811 05:15:07.633792 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:07.633799 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:07.634023 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:07.634185 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:07.634609 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.635925 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.637775 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.638673 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.638800 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:07.638968 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:07.640187 (+ 1219us) server_negotiation.cc:299] Negotiation successful
0811 05:15:07.640301 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2941,"thread_start_us":237,"threads_started":1}
I20260811 05:15:07.643815 25679 tablet_service.cc:1467] Processing CreateTablet for tablet 1bd360dde2a24a64a74728049ea2dea1 (DEFAULT_TABLE table=test-table [id=7a9657e3e2494f63b2ec98cc0c78b390]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:15:07.645110 25679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bd360dde2a24a64a74728049ea2dea1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:15:07.645968 26257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:07.630305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:44185 (local address 127.0.0.1:36144)
0811 05:15:07.634483 (+ 4178us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:07.634498 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:07.634609 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:07.636074 (+ 1465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:07.636077 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:07.636095 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:07.636262 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:07.636270 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.637775 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.637778 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:07.640353 (+ 2575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:07.640363 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.645521 (+ 5158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.645524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:07.645655 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:07.645681 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 05:15:07.645769 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 05:15:07.645839 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4111,"mutex_wait_us":367,"thread_start_us":953,"threads_started":1}
I20260811 05:15:07.646555 26252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:07.630573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36144 (local address 127.7.67.131:44185)
0811 05:15:07.633507 (+ 2934us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:07.633529 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:07.634729 (+ 1200us) server_negotiation.cc:407] Connection header received
0811 05:15:07.634815 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:07.634820 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:07.635021 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:07.635209 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:07.636399 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.637649 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.640615 (+ 2966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.641555 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.645314 (+ 3759us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:07.645465 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:07.646303 (+ 838us) server_negotiation.cc:299] Negotiation successful
0811 05:15:07.646413 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2841,"thread_start_us":306,"threads_started":1}
I20260811 05:15:07.649838 26123 tablet_service.cc:1467] Processing CreateTablet for tablet 1bd360dde2a24a64a74728049ea2dea1 (DEFAULT_TABLE table=test-table [id=7a9657e3e2494f63b2ec98cc0c78b390]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:15:07.650947 26123 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bd360dde2a24a64a74728049ea2dea1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:15:07.652652 26256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:07.632182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:39407 (local address 127.0.0.1:33904)
0811 05:15:07.639604 (+ 7422us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:07.639619 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:07.639733 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:07.647474 (+ 7741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:07.647477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:07.647497 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:07.647582 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:07.647588 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.649064 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.649067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:07.650944 (+ 1877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:07.650957 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.652207 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.652211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:07.652294 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:07.652318 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 05:15:07.652394 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 05:15:07.652464 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7364,"thread_start_us":231,"threads_started":1}
I20260811 05:15:07.653203 26259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:07.632945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33904 (local address 127.7.67.130:39407)
0811 05:15:07.636958 (+ 4013us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:07.636964 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:07.646837 (+ 9873us) server_negotiation.cc:407] Connection header received
0811 05:15:07.646931 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:07.646936 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:07.647136 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:07.647293 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:07.647738 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.648926 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.651097 (+ 2171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.651895 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.652003 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:07.652139 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:07.652925 (+ 786us) server_negotiation.cc:299] Negotiation successful
0811 05:15:07.653027 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3904,"thread_start_us":98,"threads_started":1}
I20260811 05:15:07.656693 25919 tablet_service.cc:1467] Processing CreateTablet for tablet 1bd360dde2a24a64a74728049ea2dea1 (DEFAULT_TABLE table=test-table [id=7a9657e3e2494f63b2ec98cc0c78b390]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:15:07.658078 25919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1bd360dde2a24a64a74728049ea2dea1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:15:07.669327 26260 tablet_bootstrap.cc:492] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d: Bootstrap starting.
I20260811 05:15:07.672458 26260 tablet_bootstrap.cc:654] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d: Neither blocks nor log segments found. Creating new log.
I20260811 05:15:07.673480 26260 log.cc:824] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:07.674342 26262 tablet_bootstrap.cc:492] T 1bd360dde2a24a64a74728049ea2dea1 P 220d4a0f3c884451ab1ab5c3629b22d1: Bootstrap starting.
I20260811 05:15:07.676839 26262 tablet_bootstrap.cc:654] T 1bd360dde2a24a64a74728049ea2dea1 P 220d4a0f3c884451ab1ab5c3629b22d1: Neither blocks nor log segments found. Creating new log.
I20260811 05:15:07.677809 26262 log.cc:824] T 1bd360dde2a24a64a74728049ea2dea1 P 220d4a0f3c884451ab1ab5c3629b22d1: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:07.678483 26263 tablet_bootstrap.cc:492] T 1bd360dde2a24a64a74728049ea2dea1 P ebeb41dfb7cc44c7a54e273e837de963: Bootstrap starting.
I20260811 05:15:07.680981 26260 tablet_bootstrap.cc:492] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d: No bootstrap required, opened a new log
I20260811 05:15:07.681210 26260 ts_tablet_manager.cc:1397] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.004s
I20260811 05:15:07.683323 26263 tablet_bootstrap.cc:654] T 1bd360dde2a24a64a74728049ea2dea1 P ebeb41dfb7cc44c7a54e273e837de963: Neither blocks nor log segments found. Creating new log.
I20260811 05:15:07.684856 26263 log.cc:824] T 1bd360dde2a24a64a74728049ea2dea1 P ebeb41dfb7cc44c7a54e273e837de963: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:07.688031 26260 raft_consensus.cc:348] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9127bd978bd46dfbd2a747ec3eb988d" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 44663 } } peers { permanent_uuid: "ebeb41dfb7cc44c7a54e273e837de963" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 44185 } } peers { permanent_uuid: "220d4a0f3c884451ab1ab5c3629b22d1" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39407 } }
I20260811 05:15:07.688333 26262 tablet_bootstrap.cc:492] T 1bd360dde2a24a64a74728049ea2dea1 P 220d4a0f3c884451ab1ab5c3629b22d1: No bootstrap required, opened a new log
I20260811 05:15:07.688450 26260 raft_consensus.cc:374] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:15:07.688587 26260 raft_consensus.cc:729] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9127bd978bd46dfbd2a747ec3eb988d, State: Initialized, Role: FOLLOWER
I20260811 05:15:07.688810 26262 ts_tablet_manager.cc:1397] T 1bd360dde2a24a64a74728049ea2dea1 P 220d4a0f3c884451ab1ab5c3629b22d1: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.000s
I20260811 05:15:07.689296 26260 consensus_queue.cc:260] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9127bd978bd46dfbd2a747ec3eb988d" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 44663 } } peers { permanent_uuid: "ebeb41dfb7cc44c7a54e273e837de963" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 44185 } } peers { permanent_uuid: "220d4a0f3c884451ab1ab5c3629b22d1" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39407 } }
I20260811 05:15:07.691018 26260 ts_tablet_manager.cc:1428] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d: Time spent starting tablet: real 0.010s user 0.008s sys 0.002s
I20260811 05:15:07.695811 26263 tablet_bootstrap.cc:492] T 1bd360dde2a24a64a74728049ea2dea1 P ebeb41dfb7cc44c7a54e273e837de963: No bootstrap required, opened a new log
I20260811 05:15:07.696105 26263 ts_tablet_manager.cc:1397] T 1bd360dde2a24a64a74728049ea2dea1 P ebeb41dfb7cc44c7a54e273e837de963: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.004s
I20260811 05:15:07.702078 26262 raft_consensus.cc:348] T 1bd360dde2a24a64a74728049ea2dea1 P 220d4a0f3c884451ab1ab5c3629b22d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9127bd978bd46dfbd2a747ec3eb988d" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 44663 } } peers { permanent_uuid: "ebeb41dfb7cc44c7a54e273e837de963" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 44185 } } peers { permanent_uuid: "220d4a0f3c884451ab1ab5c3629b22d1" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39407 } }
I20260811 05:15:07.702669 26262 raft_consensus.cc:374] T 1bd360dde2a24a64a74728049ea2dea1 P 220d4a0f3c884451ab1ab5c3629b22d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:15:07.702813 26262 raft_consensus.cc:729] T 1bd360dde2a24a64a74728049ea2dea1 P 220d4a0f3c884451ab1ab5c3629b22d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 220d4a0f3c884451ab1ab5c3629b22d1, State: Initialized, Role: FOLLOWER
I20260811 05:15:07.703547 26262 consensus_queue.cc:260] T 1bd360dde2a24a64a74728049ea2dea1 P 220d4a0f3c884451ab1ab5c3629b22d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9127bd978bd46dfbd2a747ec3eb988d" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 44663 } } peers { permanent_uuid: "ebeb41dfb7cc44c7a54e273e837de963" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 44185 } } peers { permanent_uuid: "220d4a0f3c884451ab1ab5c3629b22d1" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39407 } }
I20260811 05:15:07.703413 26263 raft_consensus.cc:348] T 1bd360dde2a24a64a74728049ea2dea1 P ebeb41dfb7cc44c7a54e273e837de963 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9127bd978bd46dfbd2a747ec3eb988d" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 44663 } } peers { permanent_uuid: "ebeb41dfb7cc44c7a54e273e837de963" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 44185 } } peers { permanent_uuid: "220d4a0f3c884451ab1ab5c3629b22d1" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39407 } }
I20260811 05:15:07.703722 26263 raft_consensus.cc:374] T 1bd360dde2a24a64a74728049ea2dea1 P ebeb41dfb7cc44c7a54e273e837de963 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:15:07.703874 26263 raft_consensus.cc:729] T 1bd360dde2a24a64a74728049ea2dea1 P ebeb41dfb7cc44c7a54e273e837de963 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebeb41dfb7cc44c7a54e273e837de963, State: Initialized, Role: FOLLOWER
I20260811 05:15:07.704480 26263 consensus_queue.cc:260] T 1bd360dde2a24a64a74728049ea2dea1 P ebeb41dfb7cc44c7a54e273e837de963 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9127bd978bd46dfbd2a747ec3eb988d" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 44663 } } peers { permanent_uuid: "ebeb41dfb7cc44c7a54e273e837de963" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 44185 } } peers { permanent_uuid: "220d4a0f3c884451ab1ab5c3629b22d1" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39407 } }
I20260811 05:15:07.705300 26262 ts_tablet_manager.cc:1428] T 1bd360dde2a24a64a74728049ea2dea1 P 220d4a0f3c884451ab1ab5c3629b22d1: Time spent starting tablet: real 0.016s user 0.009s sys 0.000s
I20260811 05:15:07.705876 26263 ts_tablet_manager.cc:1428] T 1bd360dde2a24a64a74728049ea2dea1 P ebeb41dfb7cc44c7a54e273e837de963: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20260811 05:15:07.705909 26021 heartbeater.cc:503] Master 127.7.67.190:41155 was elected leader, sending a full tablet report...
I20260811 05:15:07.706223 26233 heartbeater.cc:503] Master 127.7.67.190:41155 was elected leader, sending a full tablet report...
W20260811 05:15:07.771556 26022 tablet.cc:2369] T 1bd360dde2a24a64a74728049ea2dea1 P 220d4a0f3c884451ab1ab5c3629b22d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 05:15:07 dist-test-slave-04bb krb5kdc[24653](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425306, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for HTTP/127.7.67.148@KRBTEST.COM
W20260811 05:15:07.812089 26235 tablet.cc:2369] T 1bd360dde2a24a64a74728049ea2dea1 P ebeb41dfb7cc44c7a54e273e837de963: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:15:07.832883 25795 tablet.cc:2369] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:15:07.845248 26268 raft_consensus.cc:482] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:15:07.845541 26268 raft_consensus.cc:504] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9127bd978bd46dfbd2a747ec3eb988d" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 44663 } } peers { permanent_uuid: "ebeb41dfb7cc44c7a54e273e837de963" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 44185 } } peers { permanent_uuid: "220d4a0f3c884451ab1ab5c3629b22d1" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39407 } }
I20260811 05:15:07.847142 26268 leader_election.cc:288] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ebeb41dfb7cc44c7a54e273e837de963 (127.7.67.131:44185), 220d4a0f3c884451ab1ab5c3629b22d1 (127.7.67.130:39407)
I20260811 05:15:07.854024 26259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:07.848231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:44899 (local address 127.7.67.130:39407)
0811 05:15:07.848358 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:07.848362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:07.848778 (+ 416us) server_negotiation.cc:407] Connection header received
0811 05:15:07.849004 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:07.849008 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:07.849060 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:07.849106 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:07.849634 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.850458 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.852252 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.853011 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.853130 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:07.853185 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:07.853721 (+ 536us) server_negotiation.cc:299] Negotiation successful
0811 05:15:07.853843 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 05:15:07.854326 26252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:07.847612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:37995 (local address 127.7.67.131:44185)
0811 05:15:07.847800 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:07.847806 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:07.847986 (+ 180us) server_negotiation.cc:407] Connection header received
0811 05:15:07.848771 (+ 785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:07.848774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:07.848830 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:07.848878 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:07.849423 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.850190 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.852081 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.852805 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.853014 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:07.853130 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:07.853890 (+ 760us) server_negotiation.cc:299] Negotiation successful
0811 05:15:07.854081 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 05:15:07.855042 26269 raft_consensus.cc:482] T 1bd360dde2a24a64a74728049ea2dea1 P 220d4a0f3c884451ab1ab5c3629b22d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:15:07.855270 26269 raft_consensus.cc:504] T 1bd360dde2a24a64a74728049ea2dea1 P 220d4a0f3c884451ab1ab5c3629b22d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9127bd978bd46dfbd2a747ec3eb988d" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 44663 } } peers { permanent_uuid: "ebeb41dfb7cc44c7a54e273e837de963" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 44185 } } peers { permanent_uuid: "220d4a0f3c884451ab1ab5c3629b22d1" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39407 } }
I20260811 05:15:07.855360 26275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:07.847580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:39407 (local address 127.7.67.129:44899)
0811 05:15:07.848684 (+ 1104us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:07.848700 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:07.848865 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:07.849308 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:07.849310 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:07.849335 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:07.849493 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:07.849500 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.850661 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.850665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:07.852094 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:07.852107 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.853332 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.853339 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:07.853473 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:07.853506 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 05:15:07.854954 (+ 1448us) client_negotiation.cc:240] Negotiation successful
0811 05:15:07.855058 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":920,"thread_start_us":133,"threads_started":1}
I20260811 05:15:07.856724 26269 leader_election.cc:288] T 1bd360dde2a24a64a74728049ea2dea1 P 220d4a0f3c884451ab1ab5c3629b22d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b9127bd978bd46dfbd2a747ec3eb988d (127.7.67.129:44663), ebeb41dfb7cc44c7a54e273e837de963 (127.7.67.131:44185)
I20260811 05:15:07.857228 25949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bd360dde2a24a64a74728049ea2dea1" candidate_uuid: "b9127bd978bd46dfbd2a747ec3eb988d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "220d4a0f3c884451ab1ab5c3629b22d1" is_pre_election: true
I20260811 05:15:07.857461 25949 raft_consensus.cc:2454] T 1bd360dde2a24a64a74728049ea2dea1 P 220d4a0f3c884451ab1ab5c3629b22d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9127bd978bd46dfbd2a747ec3eb988d in term 0.
I20260811 05:15:07.858228 25600 leader_election.cc:302] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 220d4a0f3c884451ab1ab5c3629b22d1, b9127bd978bd46dfbd2a747ec3eb988d; no voters:
I20260811 05:15:07.858462 26274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:07.847388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:44185 (local address 127.7.67.129:37995)
0811 05:15:07.847859 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:07.847887 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:07.848603 (+ 716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:07.849023 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:07.849026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:07.849044 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:07.849284 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:07.849292 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.850442 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.850445 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:07.851939 (+ 1494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:07.851950 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.853373 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.853377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:07.853598 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:07.853645 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 05:15:07.854621 (+ 976us) client_negotiation.cc:240] Negotiation successful
0811 05:15:07.854723 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":142,"threads_started":1}
I20260811 05:15:07.858572 26268 raft_consensus.cc:2781] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:15:07.858724 26268 raft_consensus.cc:482] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:15:07.858781 26268 raft_consensus.cc:3037] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:15:07.860978 26268 raft_consensus.cc:504] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9127bd978bd46dfbd2a747ec3eb988d" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 44663 } } peers { permanent_uuid: "ebeb41dfb7cc44c7a54e273e837de963" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 44185 } } peers { permanent_uuid: "220d4a0f3c884451ab1ab5c3629b22d1" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39407 } }
I20260811 05:15:07.861420 26268 leader_election.cc:288] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d [CANDIDATE]: Term 1 election: Requested vote from peers ebeb41dfb7cc44c7a54e273e837de963 (127.7.67.131:44185), 220d4a0f3c884451ab1ab5c3629b22d1 (127.7.67.130:39407)
I20260811 05:15:07.862377 26170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bd360dde2a24a64a74728049ea2dea1" candidate_uuid: "b9127bd978bd46dfbd2a747ec3eb988d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebeb41dfb7cc44c7a54e273e837de963" is_pre_election: true
I20260811 05:15:07.862809 26170 raft_consensus.cc:2454] T 1bd360dde2a24a64a74728049ea2dea1 P ebeb41dfb7cc44c7a54e273e837de963 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b9127bd978bd46dfbd2a747ec3eb988d in term 0.
I20260811 05:15:07.862874 26168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bd360dde2a24a64a74728049ea2dea1" candidate_uuid: "b9127bd978bd46dfbd2a747ec3eb988d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebeb41dfb7cc44c7a54e273e837de963"
I20260811 05:15:07.862972 25949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bd360dde2a24a64a74728049ea2dea1" candidate_uuid: "b9127bd978bd46dfbd2a747ec3eb988d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "220d4a0f3c884451ab1ab5c3629b22d1"
I20260811 05:15:07.863003 26168 raft_consensus.cc:3037] T 1bd360dde2a24a64a74728049ea2dea1 P ebeb41dfb7cc44c7a54e273e837de963 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:15:07.863102 25949 raft_consensus.cc:3037] T 1bd360dde2a24a64a74728049ea2dea1 P 220d4a0f3c884451ab1ab5c3629b22d1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:15:07.864506 26168 raft_consensus.cc:2454] T 1bd360dde2a24a64a74728049ea2dea1 P ebeb41dfb7cc44c7a54e273e837de963 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9127bd978bd46dfbd2a747ec3eb988d in term 1.
I20260811 05:15:07.865010 25602 leader_election.cc:302] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b9127bd978bd46dfbd2a747ec3eb988d, ebeb41dfb7cc44c7a54e273e837de963; no voters:
I20260811 05:15:07.864988 25949 raft_consensus.cc:2454] T 1bd360dde2a24a64a74728049ea2dea1 P 220d4a0f3c884451ab1ab5c3629b22d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b9127bd978bd46dfbd2a747ec3eb988d in term 1.
I20260811 05:15:07.866309 26268 raft_consensus.cc:2781] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:15:07.866564 26268 raft_consensus.cc:686] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d [term 1 LEADER]: Becoming Leader. State: Replica: b9127bd978bd46dfbd2a747ec3eb988d, State: Running, Role: LEADER
I20260811 05:15:07.867081 26268 consensus_queue.cc:237] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9127bd978bd46dfbd2a747ec3eb988d" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 44663 } } peers { permanent_uuid: "ebeb41dfb7cc44c7a54e273e837de963" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 44185 } } peers { permanent_uuid: "220d4a0f3c884451ab1ab5c3629b22d1" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39407 } }
I20260811 05:15:07.873221 26252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:07.857839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:35323 (local address 127.7.67.131:44185)
0811 05:15:07.863798 (+ 5959us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:07.863802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:07.864211 (+ 409us) server_negotiation.cc:407] Connection header received
0811 05:15:07.864453 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:07.864457 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:07.864506 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:07.864588 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:07.866180 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.866911 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.871828 (+ 4917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.872425 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.872556 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:07.872617 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:07.872987 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 05:15:07.873072 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5865}
I20260811 05:15:07.874527 25435 catalog_manager.cc:5495] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d reported cstate change: term changed from 0 to 1, leader changed from <none> to b9127bd978bd46dfbd2a747ec3eb988d (127.7.67.129). New cstate: current_term: 1 leader_uuid: "b9127bd978bd46dfbd2a747ec3eb988d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9127bd978bd46dfbd2a747ec3eb988d" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 44663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ebeb41dfb7cc44c7a54e273e837de963" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 44185 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "220d4a0f3c884451ab1ab5c3629b22d1" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39407 } health_report { overall_health: UNKNOWN } } }
I20260811 05:15:07.876258 26278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:07.857259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:44185 (local address 127.7.67.130:35323)
0811 05:15:07.864058 (+ 6799us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:07.864092 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:07.864309 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:07.865239 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:07.865242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:07.865264 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:07.865560 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:07.865570 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.867070 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.867074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:07.871670 (+ 4596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:07.871681 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.872739 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.872742 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:07.872834 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:07.872860 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 05:15:07.875958 (+ 3098us) client_negotiation.cc:240] Negotiation successful
0811 05:15:07.876061 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6712,"thread_start_us":147,"threads_started":1}
I20260811 05:15:07.876962 26168 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bd360dde2a24a64a74728049ea2dea1" candidate_uuid: "220d4a0f3c884451ab1ab5c3629b22d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebeb41dfb7cc44c7a54e273e837de963" is_pre_election: true
I20260811 05:15:07.877204 26168 raft_consensus.cc:2379] T 1bd360dde2a24a64a74728049ea2dea1 P ebeb41dfb7cc44c7a54e273e837de963 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 220d4a0f3c884451ab1ab5c3629b22d1 in current term 1: Already voted for candidate b9127bd978bd46dfbd2a747ec3eb988d in this term.
I20260811 05:15:07.879683 26279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:07.857773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:44663 (local address 127.7.67.130:49661)
0811 05:15:07.865852 (+ 8079us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:07.865869 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:07.866044 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:07.868298 (+ 2254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:07.868301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:07.868318 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:07.868518 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:07.868551 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.869598 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.869601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:07.871041 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:07.871052 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.879256 (+ 8204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.879259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:07.879344 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:07.879363 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 05:15:07.879443 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 05:15:07.879526 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8003,"thread_start_us":110,"threads_started":1}
I20260811 05:15:07.879678 26253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:07.857838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:49661 (local address 127.7.67.129:44663)
0811 05:15:07.857975 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:07.857979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:07.865965 (+ 7986us) server_negotiation.cc:407] Connection header received
0811 05:15:07.868009 (+ 2044us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:07.868012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:07.868073 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:07.868126 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:07.868686 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.869409 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.878364 (+ 8955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.878992 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.879090 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:07.879157 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:07.879483 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 05:15:07.879561 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 05:15:07.880389 25686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1bd360dde2a24a64a74728049ea2dea1" candidate_uuid: "220d4a0f3c884451ab1ab5c3629b22d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9127bd978bd46dfbd2a747ec3eb988d" is_pre_election: true
I20260811 05:15:07.881266 25837 leader_election.cc:302] T 1bd360dde2a24a64a74728049ea2dea1 P 220d4a0f3c884451ab1ab5c3629b22d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 220d4a0f3c884451ab1ab5c3629b22d1; no voters: b9127bd978bd46dfbd2a747ec3eb988d, ebeb41dfb7cc44c7a54e273e837de963
I20260811 05:15:07.881860 26269 raft_consensus.cc:2726] T 1bd360dde2a24a64a74728049ea2dea1 P 220d4a0f3c884451ab1ab5c3629b22d1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 05:15:07.944792 26253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:07.939142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33178 (local address 127.7.67.129:44663)
0811 05:15:07.939273 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:07.939279 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:07.939358 (+ 79us) server_negotiation.cc:407] Connection header received
0811 05:15:07.939596 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:07.939600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:07.939675 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:07.939807 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:15:07.940200 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.941005 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.942317 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:07.942667 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:07.942780 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:07.943226 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:15:07.943865 (+ 639us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:15:07.943986 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:07.944452 (+ 466us) server_negotiation.cc:299] Negotiation successful
0811 05:15:07.944545 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 05:15:07.953807 25949 raft_consensus.cc:1261] T 1bd360dde2a24a64a74728049ea2dea1 P 220d4a0f3c884451ab1ab5c3629b22d1 [term 1 FOLLOWER]: Refusing update from remote peer b9127bd978bd46dfbd2a747ec3eb988d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:15:07.954586 26281 consensus_queue.cc:1035] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d [LEADER]: Connected to new peer: Peer: permanent_uuid: "220d4a0f3c884451ab1ab5c3629b22d1" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 39407 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:15:07.962507 26168 raft_consensus.cc:1261] T 1bd360dde2a24a64a74728049ea2dea1 P ebeb41dfb7cc44c7a54e273e837de963 [term 1 FOLLOWER]: Refusing update from remote peer b9127bd978bd46dfbd2a747ec3eb988d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:15:07.963133 26281 consensus_queue.cc:1035] T 1bd360dde2a24a64a74728049ea2dea1 P b9127bd978bd46dfbd2a747ec3eb988d [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebeb41dfb7cc44c7a54e273e837de963" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 44185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:15:07.980406 26285 mvcc.cc:205] Tried to move back new op lower bound from 7317198061368143872 to 7317198061040463872. Current Snapshot: MvccSnapshot[applied={T|T < 7317198061368143872}]
I20260811 05:15:07.986616 26293 mvcc.cc:205] Tried to move back new op lower bound from 7317198061368143872 to 7317198061040463872. Current Snapshot: MvccSnapshot[applied={T|T < 7317198061368143872}]
I20260811 05:15:07.990203 26286 mvcc.cc:205] Tried to move back new op lower bound from 7317198061368143872 to 7317198061040463872. Current Snapshot: MvccSnapshot[applied={T|T < 7317198061368143872}]
I20260811 05:15:08.001406 25435 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:15:08.001624 25435 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:15:08.004648 25435 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eee3c8c411a04a6da7d6f0e275388df6: Sending DeleteTablet for 3 replicas of tablet 1bd360dde2a24a64a74728049ea2dea1
I20260811 05:15:08.006009 26123 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bd360dde2a24a64a74728049ea2dea1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:15:07 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:36144
I20260811 05:15:08.006471 25919 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bd360dde2a24a64a74728049ea2dea1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:15:07 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:33904
I20260811 05:15:08.007184 25679 tablet_service.cc:1514] Processing DeleteTablet for tablet 1bd360dde2a24a64a74728049ea2dea1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:15:07 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:33166
I20260811 05:15:08.007411 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 25565
I20260811 05:15:08.014521 26304 tablet_replica.cc:330] T 1bd360dde2a24a64a74728049ea2dea1 P 220d4a0f3c884451ab1ab5c3629b22d1: stopping tablet replica
I20260811 05:15:08.014941 26304 raft_consensus.cc:2229] T 1bd360dde2a24a64a74728049ea2dea1 P 220d4a0f3c884451ab1ab5c3629b22d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:15:08.015288 26304 raft_consensus.cc:2258] T 1bd360dde2a24a64a74728049ea2dea1 P 220d4a0f3c884451ab1ab5c3629b22d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:15:08.020915 26303 tablet_replica.cc:330] T 1bd360dde2a24a64a74728049ea2dea1 P ebeb41dfb7cc44c7a54e273e837de963: stopping tablet replica
I20260811 05:15:08.021409 26303 raft_consensus.cc:2229] T 1bd360dde2a24a64a74728049ea2dea1 P ebeb41dfb7cc44c7a54e273e837de963 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:15:08.037204 26303 raft_consensus.cc:2258] T 1bd360dde2a24a64a74728049ea2dea1 P ebeb41dfb7cc44c7a54e273e837de963 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:15:08.039037 26303 ts_tablet_manager.cc:1905] T 1bd360dde2a24a64a74728049ea2dea1 P ebeb41dfb7cc44c7a54e273e837de963: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 05:15:08.040683 25407 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.129:44663 (error 108)
I20260811 05:15:08.042351 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 25807
I20260811 05:15:08.042415 26256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:08.042075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:44663 (local address 127.0.0.1:33192)
0811 05:15:08.042238 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:08.042321 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.129:44663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 05:15:08.047538 25407 catalog_manager.cc:4584] TS b9127bd978bd46dfbd2a747ec3eb988d (127.7.67.129:44663): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1bd360dde2a24a64a74728049ea2dea1: Network error: Client connection negotiation failed: client connection to 127.7.67.129:44663: connect: Connection refused (error 111)
I20260811 05:15:08.056921 26303 ts_tablet_manager.cc:1918] T 1bd360dde2a24a64a74728049ea2dea1 P ebeb41dfb7cc44c7a54e273e837de963: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:15:08.057127 26303 log.cc:1196] T 1bd360dde2a24a64a74728049ea2dea1 P ebeb41dfb7cc44c7a54e273e837de963: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/1bd360dde2a24a64a74728049ea2dea1
I20260811 05:15:08.057551 26303 ts_tablet_manager.cc:1939] T 1bd360dde2a24a64a74728049ea2dea1 P ebeb41dfb7cc44c7a54e273e837de963: Deleting consensus metadata
I20260811 05:15:08.059346 25412 catalog_manager.cc:4841] TS ebeb41dfb7cc44c7a54e273e837de963 (127.7.67.131:44185): tablet 1bd360dde2a24a64a74728049ea2dea1 (table test-table [id=7a9657e3e2494f63b2ec98cc0c78b390]) successfully deleted
I20260811 05:15:08.102000 26256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:08.101565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:44663 (local address 127.0.0.1:33194)
0811 05:15:08.101821 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:08.101890 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.129:44663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164}
I20260811 05:15:08.106932 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 26029
I20260811 05:15:08.116317 26256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:08.115908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:44663 (local address 127.0.0.1:33202)
0811 05:15:08.116123 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:08.116188 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.129:44663: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126}
I20260811 05:15:08.117359 26256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:08.117065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:39407 (local address 127.0.0.1:33912)
0811 05:15:08.117244 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:08.117296 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.130:39407: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 05:15:08.126483 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 25384
I20260811 05:15:08.142164 7438 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 05:15:08.487900 7438 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 05:15:08.488092 7438 mini_ranger.cc:66] Stopping Ranger...
I20260811 05:15:08.833803 7438 mini_ranger.cc:68] Stopped Ranger
2026-08-11 05:15:08.834 UTC [24703] LOG: received smart shutdown request
2026-08-11 05:15:08.840 UTC [24703] LOG: background worker "logical replication launcher" (PID 24717) exited with exit code 1
2026-08-11 05:15:08.840 UTC [24709] LOG: shutting down
2026-08-11 05:15:09.050 UTC [24703] LOG: database system is shut down
2026-08-11T05:15:09Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50644 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:15:09 dist-test-slave-04bb krb5kdc[26321](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:15:09 dist-test-slave-04bb krb5kdc[26321](info): set up 2 sockets
Aug 11 05:15:09 dist-test-slave-04bb krb5kdc[26321](info): commencing operation
krb5kdc: starting...
W20260811 05:15:10.482558 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:15:10 dist-test-slave-04bb krb5kdc[26321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425310, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:15:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:15:10Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.7.67.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.7.67.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.7.67.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/rangeradmin_127.7.67.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.7.67.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/rangeradmin_127.7.67.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.7.67.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.7.67.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.7.67.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/rangerlookup_127.7.67.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.7.67.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/rangerlookup_127.7.67.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.67.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.7.67.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.67.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/HTTP_127.7.67.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.67.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/HTTP_127.7.67.148@KRBTEST.COM.keytab.
I20260811 05:15:10.671609 7438 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasktf25Hl/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/postgres -l logfile start
2026-08-11 05:15:13.339 UTC [26371] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 05:15:13.339 UTC [26371] LOG: listening on IPv4 address "127.7.67.148", port 43211
2026-08-11 05:15:13.345 UTC [26371] LOG: listening on Unix socket "/tmp/.s.PGSQL.43211"
2026-08-11 05:15:13.368 UTC [26376] LOG: database system was shut down at 2026-08-11 05:15:11 UTC
2026-08-11 05:15:13.372 UTC [26371] LOG: database system is ready to accept connections
I20260811 05:15:14.538331 7438 mini_postgres.cc:96] Postgres bound to 43211
2026-08-11 05:15:14.546 UTC [26386] FATAL: database "slave" does not exist
127.7.67.148:43211 - accepting connections
I20260811 05:15:14.546885 7438 mini_ranger.cc:161] Starting Ranger...
I20260811 05:15:14.562964 7438 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 05:15:15.073829 7438 mini_ranger.cc:87] Created ranger Postgres database
I20260811 05:15:15.073923 7438 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ranger-admin
2026-08-11 05:15:15,362 [I] DB FLAVOR :POSTGRES
2026-08-11 05:15:15,363 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 05:15:15,363 [I] Checking connection..
2026-08-11 05:15:15,363 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:43211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 05:15:15,622 [I] Checking connection passed.
2026-08-11 05:15:15,622 [I] --------- Verifying version history table ---------
2026-08-11 05:15:15,623 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:43211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 05:15:15,872 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 05:15:15,872 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:43211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 05:15:16,135 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 05:15:16,135 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 05:15:16,135 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:43211/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 05:15:16.396 UTC [26481] WARNING: there is no transaction in progress
2026-08-11 05:15:16,406 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 05:15:16,406 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:43211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 05:15:16,664 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 05:15:16,664 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 05:15:16,664 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:43211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 05:15:16,914 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:43211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 05:15:17,159 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:43211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-04bb.c.gcp-upstream.internal','N') ;"
2026-08-11 05:15:17,406 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:43211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 05:15:17,659 [I] Table x_portal_user does not exist in database ranger
2026-08-11 05:15:17,660 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:43211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 05:15:17,903 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 05:15:17,903 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:43211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 05:15:18,166 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:43211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 05:15:18,420 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 05:15:18,420 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:43211/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 05:15:19.391 UTC [26646] WARNING: there is no transaction in progress
2026-08-11 05:15:19.408 UTC [26646] WARNING: there is no transaction in progress
2026-08-11 05:15:19.425 UTC [26646] WARNING: there is no transaction in progress
2026-08-11 05:15:19.442 UTC [26646] WARNING: there is no transaction in progress
2026-08-11 05:15:19.459 UTC [26646] WARNING: there is no transaction in progress
2026-08-11 05:15:19.478 UTC [26646] WARNING: there is no transaction in progress
2026-08-11 05:15:19.533 UTC [26646] WARNING: there is no transaction in progress
2026-08-11 05:15:19.540 UTC [26646] WARNING: there is no transaction in progress
2026-08-11 05:15:19.554 UTC [26646] WARNING: there is no transaction in progress
2026-08-11 05:15:19.564 UTC [26646] WARNING: there is no transaction in progress
2026-08-11 05:15:19.575 UTC [26646] WARNING: there is no transaction in progress
2026-08-11 05:15:19.589 UTC [26646] WARNING: there is no transaction in progress
2026-08-11 05:15:19.599 UTC [26646] WARNING: there is no transaction in progress
2026-08-11 05:15:19.606 UTC [26646] WARNING: there is no transaction in progress
2026-08-11 05:15:19.612 UTC [26646] WARNING: there is no transaction in progress
2026-08-11 05:15:19.619 UTC [26646] WARNING: there is no transaction in progress
2026-08-11 05:15:20,296 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 05:15:20,297 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:43211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 05:15:20,544 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 05:15:20,544 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:43211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 05:15:20,812 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 05:15:20,813 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:43211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 05:15:21,063 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 05:15:21,064 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:43211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 05:15:21,319 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:43211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 05:15:21,577 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:43211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-04bb.c.gcp-upstream.internal';"
2026-08-11 05:15:21,812 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 05:15:21,813 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:43211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 05:15:22,060 [I] DB_PATCHES have already been applied
I20260811 05:15:22.066260 7438 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ranger-admin:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasktf25Hl/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 05:15:22.066341 7438 mini_ranger.cc:193] Using host: 127.7.67.148
I20260811 05:15:22.070988 7438 mini_ranger.cc:227] Ranger admin URL: http://127.7.67.148:34339
Aug 11, 2026 5:15:22 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 5:15:22 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 5:15:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 5:15:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 5:15:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 5:15:22 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 5:15:22 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 05:15:23.463706 7438 mini_ranger.cc:160] Time spent starting Ranger: real 8.917s user 0.000s sys 0.012s
Aug 11, 2026 5:15:44 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 5:15:44 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 5:15:44 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 5:15:44 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 5:15:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 5:15:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 5:15:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 5:15:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 5:15:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 5:15:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 5:15:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 5:15:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 5:15:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 5:15:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 5:15:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 5:15:44 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 5:15:44 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 5:15:45 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 05:15:46.439893 7438 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.7.67.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.7.67.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.7.67.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/rangerkms_127.7.67.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.7.67.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/rangerkms_127.7.67.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.7.67.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.7.67.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.7.67.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/HTTP_127.7.67.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.7.67.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/HTTP_127.7.67.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 05:15:46 dist-test-slave-04bb krb5kdc[26321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425346, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 05:15:46.531585 7438 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 05:15:46.542557 7438 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 05:15:47.462875 7438 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 05:15:47.462970 7438 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ranger-kms
I20260811 05:15:47.462994 7438 mini_ranger_kms.cc:227] Using postgres at 127.7.67.148:43211
2026-08-11 05:15:47,665 [I] DB FLAVOR :POSTGRES
2026-08-11 05:15:47,665 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 05:15:47,665 [I] Checking connection
2026-08-11 05:15:47,665 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:43211/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 05:15:47,928 [I] connection success
2026-08-11 05:15:47,928 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 05:15:47,929 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 05:15:47,929 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:43211/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 05:15:48,194 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 05:15:48,194 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 05:15:48,195 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 05:15:48,195 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.7.67.148:43211/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 05:15:48,456 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 05:15:48.945820 7438 mini_ranger_kms.cc:310] Created kms service
I20260811 05:15:49.200234 7438 mini_ranger_kms.cc:326] Created kudu user
I20260811 05:15:49.245281 7438 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 05:15:49 dist-test-slave-04bb krb5kdc[26321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425346, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.7.67.148@KRBTEST.COM
I20260811 05:15:49.675081 7438 mini_ranger_kms.cc:384] Added ranger policy
I20260811 05:15:49.675277 7438 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ranger-kms:/tmp/dist-test-tasktf25Hl/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasktf25Hl/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 05:15:49.675315 7438 mini_ranger_kms.cc:240] Using host: 127.7.67.148
I20260811 05:15:49.680253 7438 mini_ranger_kms.cc:276] Ranger KMS PID: 26937
I20260811 05:15:49.680349 7438 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.7.67.148:50467
Aug 11, 2026 5:15:50 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 5:15:50 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 5:15:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 5:15:50 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 5:15:51 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 5:15:51 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 05:15:51 dist-test-slave-04bb krb5kdc[26321](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786425351, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.67.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 05:15:52 dist-test-slave-04bb krb5kdc[26321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425351, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.67.148@KRBTEST.COM for HTTP/127.7.67.148@KRBTEST.COM
I20260811 05:15:52.586028 7438 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.054s user 0.020s sys 0.162s
I20260811 05:15:52.586172 7438 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 05:15:52.586251 7438 mini_ranger_kms.cc:401] 127.7.67.148:50467/kms/v1/keys
Aug 11 05:15:52 dist-test-slave-04bb krb5kdc[26321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425351, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.7.67.148@KRBTEST.COM for HTTP/127.7.67.148@KRBTEST.COM
Aug 11 05:15:55 dist-test-slave-04bb krb5kdc[26321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:15:55.179996 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:36813
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:36813
--ranger_config_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.67.148:50467/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:15:55.309176 27027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:55.309432 27027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:55.309470 27027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:55.313457 27027 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:15:55.313515 27027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:55.313535 27027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:55.313551 27027 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:15:55.313566 27027 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:15:55.319260 27027 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/wal
--ranger_kms_url=127.7.67.148:50467/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:36813
--ranger_config_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:36813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:15:55.320420 27027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:55.321568 27027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:55.330711 27039 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:55.330758 27041 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:55.330716 27037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:55.330924 27027 server_base.cc:1034] running on GCE node
I20260811 05:15:55.331712 27027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:55.332960 27027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:55.334156 27027 hybrid_clock.cc:648] HybridClock initialized: now 1786425355334126 us; error 46 us; skew 500 ppm
Aug 11 05:15:55 dist-test-slave-04bb krb5kdc[26321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425355, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:55.341974 27027 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:15:55.342941 27027 webserver.cc:460] Webserver started at http://127.7.67.190:40861/ using document root <none> and password file <none>
I20260811 05:15:55.343618 27027 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:55.343703 27027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:55.343957 27027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 05:15:55 dist-test-slave-04bb krb5kdc[26321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425355, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for HTTP/127.7.67.148@KRBTEST.COM
I20260811 05:15:55.392155 27027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "54c062c181e841438dd2dd42a41eb06b"
format_stamp: "Formatted at 2026-08-11 05:15:55 on dist-test-slave-04bb"
server_key: "fb8de3945a4b5bbf606113e08a7c8784"
server_key_iv: "c6de62a7e8baa0aadb60ae6c95302216"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:55.393013 27027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "54c062c181e841438dd2dd42a41eb06b"
format_stamp: "Formatted at 2026-08-11 05:15:55 on dist-test-slave-04bb"
server_key: "fb8de3945a4b5bbf606113e08a7c8784"
server_key_iv: "c6de62a7e8baa0aadb60ae6c95302216"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:55.398098 27027 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 05:15:55.416416 27052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:55.418362 27027 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260811 05:15:55.418538 27027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "54c062c181e841438dd2dd42a41eb06b"
format_stamp: "Formatted at 2026-08-11 05:15:55 on dist-test-slave-04bb"
server_key: "fb8de3945a4b5bbf606113e08a7c8784"
server_key_iv: "c6de62a7e8baa0aadb60ae6c95302216"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:55.418735 27027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:55.441260 27027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:55.444461 27027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:55.444756 27027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:55.453510 27027 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:36813
I20260811 05:15:55.453526 27142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:36813 every 8 connection(s)
I20260811 05:15:55.454519 27027 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:15:55.457612 27143 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:15:55.458781 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 27027
I20260811 05:15:55.458917 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:15:55.459311 7438 external_mini_cluster.cc:1428] Setting key d1a7c9be706171954a4b39caa056adae
I20260811 05:15:55.464659 27143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b: Bootstrap starting.
I20260811 05:15:55.467439 27143 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b: Neither blocks nor log segments found. Creating new log.
I20260811 05:15:55.468400 27143 log.cc:824] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b: Log is configured to *not* fsync() on all Append() calls
Aug 11 05:15:55 dist-test-slave-04bb krb5kdc[26321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425355, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:15:55.470907 27143 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b: No bootstrap required, opened a new log
I20260811 05:15:55.474484 27143 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c062c181e841438dd2dd42a41eb06b" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 36813 } }
I20260811 05:15:55.474766 27143 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:15:55.474843 27143 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54c062c181e841438dd2dd42a41eb06b, State: Initialized, Role: FOLLOWER
I20260811 05:15:55.475383 27143 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c062c181e841438dd2dd42a41eb06b" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 36813 } }
I20260811 05:15:55.475544 27143 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:15:55.475629 27143 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:15:55.475757 27143 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:15:55.476996 27143 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c062c181e841438dd2dd42a41eb06b" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 36813 } }
I20260811 05:15:55.477381 27143 leader_election.cc:302] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54c062c181e841438dd2dd42a41eb06b; no voters:
I20260811 05:15:55.477737 27143 leader_election.cc:288] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:15:55.477912 27152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:15:55.477928 27149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:55.461018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42328 (local address 127.7.67.190:36813)
0811 05:15:55.461529 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:55.461548 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:55.461591 (+ 43us) server_negotiation.cc:407] Connection header received
0811 05:15:55.462434 (+ 843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:55.462460 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:55.462916 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:55.463296 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:55.464225 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:55.465666 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:55.466896 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:55.467592 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:55.467762 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:55.471013 (+ 3251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:55.471028 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:55.471046 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:55.471079 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:55.473979 (+ 2900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:55.474517 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:55.474524 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:55.474531 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:55.474676 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:55.475020 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:55.475024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:55.475026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:55.475619 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:55.475867 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:55.476348 (+ 481us) server_negotiation.cc:299] Negotiation successful
0811 05:15:55.476883 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":143,"threads_started":1}
I20260811 05:15:55.479071 27143 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:15:55.481045 27152 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [term 1 LEADER]: Becoming Leader. State: Replica: 54c062c181e841438dd2dd42a41eb06b, State: Running, Role: LEADER
I20260811 05:15:55.481477 27152 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c062c181e841438dd2dd42a41eb06b" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 36813 } }
I20260811 05:15:55.482803 27152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54c062c181e841438dd2dd42a41eb06b. Latest consensus state: current_term: 1 leader_uuid: "54c062c181e841438dd2dd42a41eb06b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c062c181e841438dd2dd42a41eb06b" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 36813 } } }
I20260811 05:15:55.483006 27152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [sys.catalog]: This master's current role is: LEADER
I20260811 05:15:55.483450 27155 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54c062c181e841438dd2dd42a41eb06b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c062c181e841438dd2dd42a41eb06b" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 36813 } } }
I20260811 05:15:55.483565 27155 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [sys.catalog]: This master's current role is: LEADER
I20260811 05:15:55.483453 27175 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:15:55.484055 27143 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 05:15:55.491046 27175 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:15:55.502774 27175 catalog_manager.cc:1347] Generated new cluster ID: 5b7e1b961b4943c780491f03ce3d6518
I20260811 05:15:55.502864 27175 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:15:55.510785 27175 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:15:55.512264 27175 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:15:55.525558 27175 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b: Generated new TSK 0
I20260811 05:15:55.527192 27175 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
I20260811 05:15:56.264168 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.67.148:50467/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:36813
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:15:56.401412 27219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:56.401728 27219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:56.401852 27219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:56.406157 27219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:56.406247 27219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:56.406400 27219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:15:56.412335 27219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.7.67.148:50467/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:36813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:15:56.413597 27219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:56.415030 27219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:56.425966 27229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:56.425976 27234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:56.426295 27232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:56.426618 27219 server_base.cc:1034] running on GCE node
I20260811 05:15:56.427078 27219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:56.427802 27219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:56.429088 27219 hybrid_clock.cc:648] HybridClock initialized: now 1786425356429061 us; error 41 us; skew 500 ppm
Aug 11 05:15:56 dist-test-slave-04bb krb5kdc[26321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425356, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:56.432199 27219 init.cc:377] Logged in from keytab as kudu/127.7.67.129@KRBTEST.COM (short username kudu)
I20260811 05:15:56.433216 27219 webserver.cc:460] Webserver started at http://127.7.67.129:36689/ using document root <none> and password file <none>
I20260811 05:15:56.433825 27219 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:56.433921 27219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:56.434206 27219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 05:15:56 dist-test-slave-04bb krb5kdc[26321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425356, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for HTTP/127.7.67.148@KRBTEST.COM
I20260811 05:15:56.451705 27219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "9d7a4bbb37924d83aa92aa0b3692ed1f"
format_stamp: "Formatted at 2026-08-11 05:15:56 on dist-test-slave-04bb"
server_key: "a2689d91c97c90726ba4387e06239cc0"
server_key_iv: "910da7c654a0bf5f241b9f5f2bdf588c"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:56.452967 27219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "9d7a4bbb37924d83aa92aa0b3692ed1f"
format_stamp: "Formatted at 2026-08-11 05:15:56 on dist-test-slave-04bb"
server_key: "a2689d91c97c90726ba4387e06239cc0"
server_key_iv: "910da7c654a0bf5f241b9f5f2bdf588c"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:56.462095 27219 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.000s
I20260811 05:15:56.493022 27242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:56.495014 27219 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.003s sys 0.000s
I20260811 05:15:56.495177 27219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "9d7a4bbb37924d83aa92aa0b3692ed1f"
format_stamp: "Formatted at 2026-08-11 05:15:56 on dist-test-slave-04bb"
server_key: "a2689d91c97c90726ba4387e06239cc0"
server_key_iv: "910da7c654a0bf5f241b9f5f2bdf588c"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:56.495337 27219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:56.513798 27219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:56.517730 27219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:56.518021 27219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:56.518857 27219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:56.519996 27219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:15:56.520104 27219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:56.520249 27219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:15:56.520337 27219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:56.536279 27219 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:45913
I20260811 05:15:56.537650 27219 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:15:56.539669 27413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:45913 every 8 connection(s)
I20260811 05:15:56.545433 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 27219
I20260811 05:15:56.545564 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:15:56.545931 7438 external_mini_cluster.cc:1428] Setting key 8842b7bbe356ba58418e12542c09b6ea
Aug 11 05:15:56 dist-test-slave-04bb krb5kdc[26321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425356, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:15:56.559129 27419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:56.542508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:34645 (local address 127.7.67.190:36813)
0811 05:15:56.543055 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:56.543060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:56.543584 (+ 524us) server_negotiation.cc:407] Connection header received
0811 05:15:56.545023 (+ 1439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:56.545027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:56.545090 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:56.545198 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:56.547013 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:56.547905 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:56.549687 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:56.550105 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:56.550186 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:56.553615 (+ 3429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:56.553619 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:56.553622 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:56.553651 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:56.556181 (+ 2530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:56.556975 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:56.556979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:56.556981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:56.557084 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:56.557575 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:56.557578 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:56.557580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:56.557861 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:56.557969 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:56.558772 (+ 803us) server_negotiation.cc:299] Negotiation successful
0811 05:15:56.558929 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":115,"threads_started":1}
WARNING: no policy specified for kudu/127.7.67.130@KRBTEST.COM; defaulting to no policy
I20260811 05:15:56.561344 27421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:56.542789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:36813 (local address 127.7.67.129:34645)
0811 05:15:56.543403 (+ 614us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:56.543470 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:56.544739 (+ 1269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:56.545528 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:56.545537 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:56.545904 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:56.546834 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:56.546861 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:56.548069 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:56.548072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:56.549540 (+ 1468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:56.549551 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:56.550423 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:56.550426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:56.550630 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:56.551379 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:56.551401 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:56.553385 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:56.556343 (+ 2958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:56.556351 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:56.556372 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:56.556835 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:56.557211 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:56.557214 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:56.557216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:56.557429 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:56.558047 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:56.558054 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:56.558428 (+ 374us) client_negotiation.cc:769] Sending connection context
0811 05:15:56.559660 (+ 1232us) client_negotiation.cc:240] Negotiation successful
0811 05:15:56.560074 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":132,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.130@KRBTEST.COM" created.
I20260811 05:15:56.563213 27415 heartbeater.cc:347] Connected to a master server at 127.7.67.190:36813
I20260811 05:15:56.563473 27415 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:56.564093 27415 heartbeater.cc:511] Master 127.7.67.190:36813 requested a full tablet report, sending...
I20260811 05:15:56.566052 27082 ts_manager.cc:194] Registered new tserver with Master: 9d7a4bbb37924d83aa92aa0b3692ed1f (127.7.67.129:45913)
I20260811 05:15:56.567703 27082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.129@KRBTEST.COM'} at 127.7.67.129:34645
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
I20260811 05:15:56.587855 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.67.148:50467/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:36813
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:15:56 dist-test-slave-04bb krb5kdc[26321](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786425356, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 05:15:56.721071 27432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:56.721369 27432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:56.721489 27432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:56.725955 27432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:56.726032 27432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:56.726142 27432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:15:56.731976 27432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.7.67.148:50467/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:36813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:15:56.733371 27432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:56.734704 27432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:56.746840 27445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:56.747067 27442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:56.747419 27441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:56.748340 27432 server_base.cc:1034] running on GCE node
I20260811 05:15:56.748996 27432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:56.749825 27432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:56.750982 27432 hybrid_clock.cc:648] HybridClock initialized: now 1786425356750967 us; error 31 us; skew 500 ppm
Aug 11 05:15:56 dist-test-slave-04bb krb5kdc[26321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425356, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:56.754870 27432 init.cc:377] Logged in from keytab as kudu/127.7.67.130@KRBTEST.COM (short username kudu)
I20260811 05:15:56.756095 27432 webserver.cc:460] Webserver started at http://127.7.67.130:41671/ using document root <none> and password file <none>
I20260811 05:15:56.756958 27432 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:56.757110 27432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:56.757452 27432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 05:15:56 dist-test-slave-04bb krb5kdc[26321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425356, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for HTTP/127.7.67.148@KRBTEST.COM
I20260811 05:15:56.776280 27432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "df8ff797347f4ef09a5d8e7925f70c01"
format_stamp: "Formatted at 2026-08-11 05:15:56 on dist-test-slave-04bb"
server_key: "1720b52e7c99e9780c259c53bf2d8c13"
server_key_iv: "d49b639b9c9b38c682c399edbc95c4f4"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:56.777086 27432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "df8ff797347f4ef09a5d8e7925f70c01"
format_stamp: "Formatted at 2026-08-11 05:15:56 on dist-test-slave-04bb"
server_key: "1720b52e7c99e9780c259c53bf2d8c13"
server_key_iv: "d49b639b9c9b38c682c399edbc95c4f4"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:56.782606 27432 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 05:15:56.805874 27454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:56.807756 27432 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.001s
I20260811 05:15:56.807911 27432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "df8ff797347f4ef09a5d8e7925f70c01"
format_stamp: "Formatted at 2026-08-11 05:15:56 on dist-test-slave-04bb"
server_key: "1720b52e7c99e9780c259c53bf2d8c13"
server_key_iv: "d49b639b9c9b38c682c399edbc95c4f4"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:56.808066 27432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:56.840899 27432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:56.844054 27432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:56.844269 27432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:56.844976 27432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:56.845885 27432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:15:56.845933 27432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:56.845999 27432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:15:56.846035 27432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:56.857206 27432 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:44797
I20260811 05:15:56.857311 27644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:44797 every 8 connection(s)
I20260811 05:15:56.858225 27432 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:15:56.867136 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 27432
I20260811 05:15:56.867273 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:15:56.867591 7438 external_mini_cluster.cc:1428] Setting key 3d0a9f0456b3c352260fb6799507a639
Aug 11 05:15:56 dist-test-slave-04bb krb5kdc[26321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425356, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:15:56.883627 27419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:56.860228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:56755 (local address 127.7.67.190:36813)
0811 05:15:56.860418 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:56.860425 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:56.861469 (+ 1044us) server_negotiation.cc:407] Connection header received
0811 05:15:56.862971 (+ 1502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:56.862974 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:56.863024 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:56.863103 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:56.864864 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:56.865379 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:56.868794 (+ 3415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:56.869166 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:56.869263 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:56.877656 (+ 8393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:56.877673 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:56.877676 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:56.877712 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:56.879736 (+ 2024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:56.881041 (+ 1305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:56.881045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:56.881047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:56.881149 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:56.882141 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:56.882145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:56.882147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:56.882418 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:56.882509 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:56.883283 (+ 774us) server_negotiation.cc:299] Negotiation successful
0811 05:15:56.883447 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 05:15:56.885322 27651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:56.860658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:36813 (local address 127.7.67.130:56755)
0811 05:15:56.861248 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:56.861326 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:56.862645 (+ 1319us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:56.863338 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:56.863351 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:56.863794 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:56.864674 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:56.864702 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:56.865512 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:56.865516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:56.868431 (+ 2915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:56.868443 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:56.869488 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:56.869492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:56.869711 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:56.870445 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:56.870467 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:56.877400 (+ 6933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:56.879933 (+ 2533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:56.879941 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:56.879962 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:56.880377 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:56.881278 (+ 901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:56.881281 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:56.881283 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:56.881489 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:56.882584 (+ 1095us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:56.882591 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:56.882970 (+ 379us) client_negotiation.cc:769] Sending connection context
0811 05:15:56.883820 (+ 850us) client_negotiation.cc:240] Negotiation successful
0811 05:15:56.884195 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":124,"threads_started":1}
I20260811 05:15:56.890259 27648 heartbeater.cc:347] Connected to a master server at 127.7.67.190:36813
I20260811 05:15:56.890556 27648 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:56.891249 27648 heartbeater.cc:511] Master 127.7.67.190:36813 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.67.131@KRBTEST.COM; defaulting to no policy
I20260811 05:15:56.892650 27082 ts_manager.cc:194] Registered new tserver with Master: df8ff797347f4ef09a5d8e7925f70c01 (127.7.67.130:44797)
I20260811 05:15:56.893476 27082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.130@KRBTEST.COM'} at 127.7.67.130:56755
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
I20260811 05:15:56.920785 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.67.148:50467/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:36813
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:15:57 dist-test-slave-04bb krb5kdc[26321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425356, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for HTTP/127.7.67.148@KRBTEST.COM
W20260811 05:15:57.091239 27655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:57.091627 27655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:57.091706 27655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:57.098377 27655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:57.098462 27655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:57.098548 27655 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:15:57.106318 27655 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.7.67.148:50467/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:36813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27655
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:15:57.107738 27655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:57.109320 27655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:57.122116 27667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:57.122116 27669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:57.122747 27666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:57.123999 27655 server_base.cc:1034] running on GCE node
I20260811 05:15:57.124702 27655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:57.125504 27655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:57.126786 27655 hybrid_clock.cc:648] HybridClock initialized: now 1786425357126721 us; error 82 us; skew 500 ppm
Aug 11 05:15:57 dist-test-slave-04bb krb5kdc[26321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425357, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:57.130647 27655 init.cc:377] Logged in from keytab as kudu/127.7.67.131@KRBTEST.COM (short username kudu)
I20260811 05:15:57.131841 27655 webserver.cc:460] Webserver started at http://127.7.67.131:33725/ using document root <none> and password file <none>
I20260811 05:15:57.132625 27655 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:57.132736 27655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:15:57.132992 27655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 05:15:57 dist-test-slave-04bb krb5kdc[26321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425357, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for HTTP/127.7.67.148@KRBTEST.COM
I20260811 05:15:57.150861 27655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "bdb85032b7a44bfea527bc1d5d7ef7da"
format_stamp: "Formatted at 2026-08-11 05:15:57 on dist-test-slave-04bb"
server_key: "ff0a8215b7be77409297ac86c8579340"
server_key_iv: "02e45f0e356ce3bb6c518dcec66574ac"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:57.151644 27655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "bdb85032b7a44bfea527bc1d5d7ef7da"
format_stamp: "Formatted at 2026-08-11 05:15:57 on dist-test-slave-04bb"
server_key: "ff0a8215b7be77409297ac86c8579340"
server_key_iv: "02e45f0e356ce3bb6c518dcec66574ac"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:57.157076 27655 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 05:15:57.180080 27680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:57.185902 27655 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.004s sys 0.000s
I20260811 05:15:57.186079 27655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "bdb85032b7a44bfea527bc1d5d7ef7da"
format_stamp: "Formatted at 2026-08-11 05:15:57 on dist-test-slave-04bb"
server_key: "ff0a8215b7be77409297ac86c8579340"
server_key_iv: "02e45f0e356ce3bb6c518dcec66574ac"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:57.186236 27655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:57.233276 27655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:57.237054 27655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:57.237253 27655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:57.237957 27655 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:57.238984 27655 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:15:57.239048 27655 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:57.239102 27655 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:15:57.239130 27655 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:57.256345 27655 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:45731
I20260811 05:15:57.256376 27859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:45731 every 8 connection(s)
I20260811 05:15:57.257887 27655 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:15:57.263227 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 27655
I20260811 05:15:57.263371 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:15:57.263756 7438 external_mini_cluster.cc:1428] Setting key d520a83f9d945d6ab8bd86ace27db96a
Aug 11 05:15:57 dist-test-slave-04bb krb5kdc[26321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425357, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:15:57.276262 27419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:57.262437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:33797 (local address 127.7.67.190:36813)
0811 05:15:57.262589 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:57.262594 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:57.262614 (+ 20us) server_negotiation.cc:407] Connection header received
0811 05:15:57.263186 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:57.263190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:57.263246 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:57.263301 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:57.264980 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:57.265784 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:57.267666 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:57.268088 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:57.268234 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:57.271409 (+ 3175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:57.271415 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:57.271418 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:57.271454 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:57.273500 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:57.274158 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:57.274163 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:57.274165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:57.274285 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:57.274825 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:57.274829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:57.274831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:57.275107 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:57.275245 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:57.275903 (+ 658us) server_negotiation.cc:299] Negotiation successful
0811 05:15:57.276064 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 05:15:57.277148 27863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:57.260978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:36813 (local address 127.7.67.131:33797)
0811 05:15:57.261755 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:57.261819 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:57.262926 (+ 1107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:57.263735 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:57.263744 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:57.264094 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:57.264809 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:57.264831 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:57.265948 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:57.265951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:57.267498 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:57.267510 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:57.268283 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:57.268287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:57.268484 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:57.269307 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:57.269329 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:57.271204 (+ 1875us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:57.273663 (+ 2459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:57.273669 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:57.273686 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:57.274011 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:57.274407 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:57.274411 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:57.274413 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:57.274677 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:57.275240 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:57.275247 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:57.275565 (+ 318us) client_negotiation.cc:769] Sending connection context
0811 05:15:57.275776 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 05:15:57.276092 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":300,"threads_started":1}
I20260811 05:15:57.279172 27860 heartbeater.cc:347] Connected to a master server at 127.7.67.190:36813
I20260811 05:15:57.279429 27860 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:57.280019 27860 heartbeater.cc:511] Master 127.7.67.190:36813 requested a full tablet report, sending...
I20260811 05:15:57.281250 27081 ts_manager.cc:194] Registered new tserver with Master: bdb85032b7a44bfea527bc1d5d7ef7da (127.7.67.131:45731)
I20260811 05:15:57.281965 27081 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.131@KRBTEST.COM'} at 127.7.67.131:33797
I20260811 05:15:57.292102 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:15:57.293077 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 27219
I20260811 05:15:57.316705 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 27432
I20260811 05:15:57.330405 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 27655
I20260811 05:15:57.347047 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 27027
I20260811 05:15:57.362052 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:36813
--webserver_interface=127.7.67.190
--webserver_port=40861
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:36813
--ranger_config_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.67.148:50467/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.67.148:50467/kms with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:15:57.478227 27865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:57.478547 27865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:57.478636 27865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:57.482796 27865 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:15:57.482897 27865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:57.482935 27865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:57.482997 27865 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:15:57.483043 27865 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:15:57.488853 27865 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.67.148:50467/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:36813
--ranger_config_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:36813
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=40861
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:15:57.490036 27865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:57.491179 27865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:57.500065 27880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:57.500057 27874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:57.500061 27877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:57.500437 27865 server_base.cc:1034] running on GCE node
I20260811 05:15:57.501070 27865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:57.502164 27865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:57.503369 27865 hybrid_clock.cc:648] HybridClock initialized: now 1786425357503331 us; error 27 us; skew 500 ppm
Aug 11 05:15:57 dist-test-slave-04bb krb5kdc[26321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425357, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:57.506238 27865 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:15:57.507092 27865 webserver.cc:460] Webserver started at http://127.7.67.190:40861/ using document root <none> and password file <none>
I20260811 05:15:57.507608 27865 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:57.507740 27865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 05:15:57 dist-test-slave-04bb krb5kdc[26321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425357, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for HTTP/127.7.67.148@KRBTEST.COM
I20260811 05:15:57.595484 27865 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:15:57.598018 27896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:57.599390 27865 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 05:15:57.599540 27865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "54c062c181e841438dd2dd42a41eb06b"
format_stamp: "Formatted at 2026-08-11 05:15:55 on dist-test-slave-04bb"
server_key: "fb8de3945a4b5bbf606113e08a7c8784"
server_key_iv: "c6de62a7e8baa0aadb60ae6c95302216"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:57.600116 27865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:57.611521 27865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:57.614243 27865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:57.614449 27865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:57.621886 27865 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:36813
I20260811 05:15:57.621922 27979 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:36813 every 8 connection(s)
I20260811 05:15:57.622997 27865 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:15:57.627550 27980 sys_catalog.cc:263] Verifying existing consensus state
I20260811 05:15:57.629410 27980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b: Bootstrap starting.
I20260811 05:15:57.631115 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 27865
I20260811 05:15:57.631624 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.67.148:50467/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:45913
--local_ip_for_outbound_sockets=127.7.67.129
--tserver_master_addrs=127.7.67.190:36813
--webserver_port=36689
--webserver_interface=127.7.67.129
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.67.148:50467/kms with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:15:57.647029 27980 log.cc:824] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:57.653960 27980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:15:57.654376 27980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b: Bootstrap complete.
I20260811 05:15:57.658634 27980 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c062c181e841438dd2dd42a41eb06b" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 36813 } }
I20260811 05:15:57.659024 27980 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54c062c181e841438dd2dd42a41eb06b, State: Initialized, Role: FOLLOWER
I20260811 05:15:57.659528 27980 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c062c181e841438dd2dd42a41eb06b" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 36813 } }
I20260811 05:15:57.659641 27980 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:15:57.659727 27980 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:15:57.659816 27980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [term 1 FOLLOWER]: Advancing to term 2
I20260811 05:15:57.661734 27980 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c062c181e841438dd2dd42a41eb06b" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 36813 } }
I20260811 05:15:57.662083 27980 leader_election.cc:302] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54c062c181e841438dd2dd42a41eb06b; no voters:
I20260811 05:15:57.662421 27980 leader_election.cc:288] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 05:15:57.662508 27985 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [term 2 FOLLOWER]: Leader election won for term 2
I20260811 05:15:57.662699 27985 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [term 2 LEADER]: Becoming Leader. State: Replica: 54c062c181e841438dd2dd42a41eb06b, State: Running, Role: LEADER
I20260811 05:15:57.663022 27985 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c062c181e841438dd2dd42a41eb06b" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 36813 } }
I20260811 05:15:57.663689 27980 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:15:57.664011 27985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54c062c181e841438dd2dd42a41eb06b. Latest consensus state: current_term: 2 leader_uuid: "54c062c181e841438dd2dd42a41eb06b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c062c181e841438dd2dd42a41eb06b" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 36813 } } }
I20260811 05:15:57.664134 27985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [sys.catalog]: This master's current role is: LEADER
I20260811 05:15:57.664566 27992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:15:57.664588 27986 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "54c062c181e841438dd2dd42a41eb06b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54c062c181e841438dd2dd42a41eb06b" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 36813 } } }
I20260811 05:15:57.664808 27986 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b [sys.catalog]: This master's current role is: LEADER
I20260811 05:15:57.668160 27992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:15:57.668715 27980 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 05:15:57.668926 27992 catalog_manager.cc:1259] Loaded cluster ID: 5b7e1b961b4943c780491f03ce3d6518
I20260811 05:15:57.668982 27992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:15:57.671294 27992 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:15:57.672082 27992 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b: Loaded TSK: 0
I20260811 05:15:57.673569 27992 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 05:15:57.779821 27982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:57.780162 27982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:57.780227 27982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:57.786119 27982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:57.786201 27982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:57.786288 27982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:15:57.793924 27982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.67.148:50467/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:45913
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=36689
--tserver_master_addrs=127.7.67.190:36813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:15:57.795269 27982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:57.796639 27982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:57.809424 28045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:57.809973 28040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:57.810322 27982 server_base.cc:1034] running on GCE node
W20260811 05:15:57.810580 28039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:57.811107 27982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:57.811909 27982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:57.813104 27982 hybrid_clock.cc:648] HybridClock initialized: now 1786425357813061 us; error 54 us; skew 500 ppm
Aug 11 05:15:57 dist-test-slave-04bb krb5kdc[26321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425357, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:57.817068 27982 init.cc:377] Logged in from keytab as kudu/127.7.67.129@KRBTEST.COM (short username kudu)
I20260811 05:15:57.818192 27982 webserver.cc:460] Webserver started at http://127.7.67.129:36689/ using document root <none> and password file <none>
I20260811 05:15:57.818924 27982 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:57.819015 27982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 05:15:57 dist-test-slave-04bb krb5kdc[26321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425357, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for HTTP/127.7.67.148@KRBTEST.COM
I20260811 05:15:57.844342 27982 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.003s sys 0.000s
I20260811 05:15:57.850381 28058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:57.852049 27982 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260811 05:15:57.852185 27982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "9d7a4bbb37924d83aa92aa0b3692ed1f"
format_stamp: "Formatted at 2026-08-11 05:15:56 on dist-test-slave-04bb"
server_key: "a2689d91c97c90726ba4387e06239cc0"
server_key_iv: "910da7c654a0bf5f241b9f5f2bdf588c"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:57.852967 27982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:57.884625 27982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:57.888567 27982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:57.888787 27982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:57.892874 27982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:57.894331 27982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:15:57.894411 27982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:57.894490 27982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:15:57.894527 27982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:57.907298 27982 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:45913
I20260811 05:15:57.907416 28245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:45913 every 8 connection(s)
I20260811 05:15:57.908839 27982 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:15:57.918463 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 27982
I20260811 05:15:57.918959 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.67.148:50467/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:44797
--local_ip_for_outbound_sockets=127.7.67.130
--tserver_master_addrs=127.7.67.190:36813
--webserver_port=41671
--webserver_interface=127.7.67.130
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.67.148:50467/kms with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:15:57 dist-test-slave-04bb krb5kdc[26321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425357, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:15:57.931942 28252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:57.911537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:33173 (local address 127.7.67.190:36813)
0811 05:15:57.912195 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:57.912218 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:57.912729 (+ 511us) server_negotiation.cc:407] Connection header received
0811 05:15:57.914228 (+ 1499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:57.914244 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:57.914637 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:57.914980 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:57.916793 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:57.918546 (+ 1753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:57.920469 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:57.921322 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:57.921478 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:57.925018 (+ 3540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:57.925026 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:57.925045 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:57.925082 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:57.927557 (+ 2475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:57.928352 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:57.928359 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:57.928366 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:57.928555 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:57.929017 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:57.929021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:57.929024 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:57.929539 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:57.929669 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:57.930469 (+ 800us) server_negotiation.cc:299] Negotiation successful
0811 05:15:57.930803 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":133,"threads_started":1}
I20260811 05:15:57.933887 28253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:57.911706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:36813 (local address 127.7.67.129:33173)
0811 05:15:57.912513 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:57.912605 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:57.913719 (+ 1114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:57.915570 (+ 1851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:57.915579 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:57.915929 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:57.916593 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:57.916613 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:57.918738 (+ 2125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:57.918743 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:57.920304 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:57.920317 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:57.921762 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:57.921770 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:57.921980 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:57.922763 (+ 783us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:57.922787 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:57.924758 (+ 1971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:57.927717 (+ 2959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:57.927725 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:57.927743 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:57.928184 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:57.928688 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:57.928692 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:57.928694 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:57.928889 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:57.929750 (+ 861us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:57.929757 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:57.930067 (+ 310us) client_negotiation.cc:769] Sending connection context
0811 05:15:57.932505 (+ 2438us) client_negotiation.cc:240] Negotiation successful
0811 05:15:57.932881 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":163,"threads_started":1}
I20260811 05:15:57.936935 28246 heartbeater.cc:347] Connected to a master server at 127.7.67.190:36813
I20260811 05:15:57.937217 28246 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:57.937785 28246 heartbeater.cc:511] Master 127.7.67.190:36813 requested a full tablet report, sending...
W20260811 05:15:57.940066 28246 heartbeater.cc:644] Failed to heartbeat to 127.7.67.190:36813 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 05:15:57.940225 28246 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:57.940438 28246 heartbeater.cc:511] Master 127.7.67.190:36813 requested a full tablet report, sending...
I20260811 05:15:57.941460 28246 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:57.941663 28246 heartbeater.cc:511] Master 127.7.67.190:36813 requested a full tablet report, sending...
W20260811 05:15:57.942533 28246 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 05:15:58.077610 28254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:58.077863 28254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:58.077984 28254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:58.083046 28254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:58.083108 28254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:58.083217 28254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:15:58.088881 28254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.67.148:50467/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:44797
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=41671
--tserver_master_addrs=127.7.67.190:36813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:15:58.090080 28254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:58.091295 28254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:58.110103 28266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:58.110440 28263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:58.110617 28268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:58.110642 28254 server_base.cc:1034] running on GCE node
I20260811 05:15:58.111415 28254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:58.112234 28254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:58.115542 28254 hybrid_clock.cc:648] HybridClock initialized: now 1786425358115498 us; error 53 us; skew 500 ppm
Aug 11 05:15:58 dist-test-slave-04bb krb5kdc[26321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425358, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:58.119022 28254 init.cc:377] Logged in from keytab as kudu/127.7.67.130@KRBTEST.COM (short username kudu)
I20260811 05:15:58.120092 28254 webserver.cc:460] Webserver started at http://127.7.67.130:41671/ using document root <none> and password file <none>
I20260811 05:15:58.120765 28254 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:58.120880 28254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 05:15:58 dist-test-slave-04bb krb5kdc[26321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425358, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for HTTP/127.7.67.148@KRBTEST.COM
I20260811 05:15:58.145360 28254 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 05:15:58.148079 28282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:58.149581 28254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 05:15:58.149724 28254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "df8ff797347f4ef09a5d8e7925f70c01"
format_stamp: "Formatted at 2026-08-11 05:15:56 on dist-test-slave-04bb"
server_key: "1720b52e7c99e9780c259c53bf2d8c13"
server_key_iv: "d49b639b9c9b38c682c399edbc95c4f4"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:58.150323 28254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:58.208254 28254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:58.212399 28254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:58.212673 28254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:58.213524 28254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:58.214671 28254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:15:58.214741 28254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:58.214824 28254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:15:58.214859 28254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:58.228822 28254 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:44797
I20260811 05:15:58.228914 28486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:44797 every 8 connection(s)
I20260811 05:15:58.230350 28254 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
Aug 11 05:15:58 dist-test-slave-04bb krb5kdc[26321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425358, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:15:58.240654 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 28254
I20260811 05:15:58.241133 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.7.67.148:50467/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:45731
--local_ip_for_outbound_sockets=127.7.67.131
--tserver_master_addrs=127.7.67.190:36813
--webserver_port=33725
--webserver_interface=127.7.67.131
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.7.67.148:50467/kms with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:15:58.252789 28252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:58.232387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:45487 (local address 127.7.67.190:36813)
0811 05:15:58.232581 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:58.232586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:58.233502 (+ 916us) server_negotiation.cc:407] Connection header received
0811 05:15:58.234695 (+ 1193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:58.234698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:58.234760 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:58.234856 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:58.235921 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:58.236763 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:58.238059 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:58.238398 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:58.238516 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:58.248180 (+ 9664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:58.248184 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:58.248187 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:58.248223 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:58.250205 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:58.250955 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:58.250959 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:58.250961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:58.251054 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:58.251493 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:58.251497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:58.251499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:58.251715 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:58.251784 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:58.252409 (+ 625us) server_negotiation.cc:299] Negotiation successful
0811 05:15:58.252600 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 05:15:58.253675 28489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:58.232857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:36813 (local address 127.7.67.130:45487)
0811 05:15:58.233357 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:58.233410 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:58.234459 (+ 1049us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:58.234995 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:58.235002 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:58.235280 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:58.235760 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:58.235774 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:58.236895 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:58.236897 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:58.237890 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:58.237900 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:58.238564 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:58.238567 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:58.238714 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:58.239333 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:58.239350 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:58.247905 (+ 8555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:58.250385 (+ 2480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:58.250392 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:58.250412 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:58.250822 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:58.251172 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:58.251176 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:58.251178 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:58.251373 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:58.251859 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:58.251864 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:58.252142 (+ 278us) client_negotiation.cc:769] Sending connection context
0811 05:15:58.252368 (+ 226us) client_negotiation.cc:240] Negotiation successful
0811 05:15:58.252781 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":143,"threads_started":1}
I20260811 05:15:58.255004 28487 heartbeater.cc:347] Connected to a master server at 127.7.67.190:36813
I20260811 05:15:58.255362 28487 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:58.255903 28487 heartbeater.cc:511] Master 127.7.67.190:36813 requested a full tablet report, sending...
W20260811 05:15:58.269641 28487 heartbeater.cc:644] Failed to heartbeat to 127.7.67.190:36813 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 05:15:58.269845 28487 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:58.270099 28487 heartbeater.cc:511] Master 127.7.67.190:36813 requested a full tablet report, sending...
I20260811 05:15:58.271056 28487 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:58.271250 28487 heartbeater.cc:511] Master 127.7.67.190:36813 requested a full tablet report, sending...
W20260811 05:15:58.272083 28487 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 05:15:58.402192 28490 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:15:58.402542 28490 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:15:58.402614 28490 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:15:58.407560 28490 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:15:58.407642 28490 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:15:58.407747 28490 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:15:58.413878 28490 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.7.67.148:50467/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:45731
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=33725
--tserver_master_addrs=127.7.67.190:36813
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28490
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:15:58.415009 28490 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:15:58.416124 28490 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:15:58.428978 28499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:58.429168 28502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:15:58.429529 28498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:15:58.430713 28490 server_base.cc:1034] running on GCE node
I20260811 05:15:58.431205 28490 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:15:58.432034 28490 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:15:58.433233 28490 hybrid_clock.cc:648] HybridClock initialized: now 1786425358433198 us; error 49 us; skew 500 ppm
Aug 11 05:15:58 dist-test-slave-04bb krb5kdc[26321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425358, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:58.436843 28490 init.cc:377] Logged in from keytab as kudu/127.7.67.131@KRBTEST.COM (short username kudu)
I20260811 05:15:58.437933 28490 webserver.cc:460] Webserver started at http://127.7.67.131:33725/ using document root <none> and password file <none>
I20260811 05:15:58.438652 28490 fs_manager.cc:359] Metadata directory not provided
I20260811 05:15:58.438778 28490 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 05:15:58 dist-test-slave-04bb krb5kdc[26321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425358, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for HTTP/127.7.67.148@KRBTEST.COM
I20260811 05:15:58.470187 28490 fs_manager.cc:709] Time spent opening directory manager: real 0.016s user 0.003s sys 0.000s
I20260811 05:15:58.477499 28525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:58.479280 28490 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.004s sys 0.000s
I20260811 05:15:58.479426 28490 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "bdb85032b7a44bfea527bc1d5d7ef7da"
format_stamp: "Formatted at 2026-08-11 05:15:57 on dist-test-slave-04bb"
server_key: "ff0a8215b7be77409297ac86c8579340"
server_key_iv: "02e45f0e356ce3bb6c518dcec66574ac"
server_key_version: "kuduclusterkey@0"
I20260811 05:15:58.480221 28490 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:15:58.515448 28490 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:15:58.519181 28490 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:15:58.519392 28490 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:15:58.520138 28490 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:15:58.521307 28490 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:15:58.521360 28490 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:58.521414 28490 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:15:58.521435 28490 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:15:58.533133 28490 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:45731
I20260811 05:15:58.534475 28490 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:15:58.537058 28722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:45731 every 8 connection(s)
I20260811 05:15:58.542223 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 28490
Aug 11 05:15:58 dist-test-slave-04bb krb5kdc[26321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425358, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:15:58.554040 28252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:58.538966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:47417 (local address 127.7.67.190:36813)
0811 05:15:58.539114 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:58.539119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:58.540304 (+ 1185us) server_negotiation.cc:407] Connection header received
0811 05:15:58.541614 (+ 1310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:58.541618 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:58.541680 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:58.541736 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:58.543817 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:58.544651 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:58.546349 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:58.546701 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:58.546784 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:58.549896 (+ 3112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:58.549900 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:58.549903 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:58.549932 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:58.551662 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:58.552264 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:58.552268 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:58.552270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:58.552352 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:58.552773 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:58.552776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:58.552778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:58.552994 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:58.553062 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:58.553745 (+ 683us) server_negotiation.cc:299] Negotiation successful
0811 05:15:58.553885 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 05:15:58.555995 28727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:58.539483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:36813 (local address 127.7.67.131:47417)
0811 05:15:58.540126 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:58.540189 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:58.541351 (+ 1162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:58.542171 (+ 820us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:58.542179 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:58.542513 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:15:58.543641 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:58.543661 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:58.544810 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:58.544814 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:58.546201 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:58.546213 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:58.547010 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:58.547014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:58.547209 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:58.547965 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:15:58.547987 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:15:58.549658 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:15:58.551793 (+ 2135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:58.551799 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:58.551817 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:58.552149 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:58.552456 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:15:58.552459 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:15:58.552461 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:15:58.552663 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:15:58.553143 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:15:58.553150 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:15:58.553464 (+ 314us) client_negotiation.cc:769] Sending connection context
0811 05:15:58.554472 (+ 1008us) client_negotiation.cc:240] Negotiation successful
0811 05:15:58.554830 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":169,"threads_started":1}
I20260811 05:15:58.558611 28724 heartbeater.cc:347] Connected to a master server at 127.7.67.190:36813
I20260811 05:15:58.558890 28724 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:58.559490 28724 heartbeater.cc:511] Master 127.7.67.190:36813 requested a full tablet report, sending...
I20260811 05:15:58.561167 27919 ts_manager.cc:194] Registered new tserver with Master: bdb85032b7a44bfea527bc1d5d7ef7da (127.7.67.131:45731)
I20260811 05:15:58.562709 27919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.131@KRBTEST.COM'} at 127.7.67.131:47417
I20260811 05:15:58.573239 28729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:58.542898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42338 (local address 127.7.67.190:36813)
0811 05:15:58.547664 (+ 4766us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:58.547673 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:58.550238 (+ 2565us) server_negotiation.cc:407] Connection header received
0811 05:15:58.550281 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:58.550284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:58.550335 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:58.550449 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:58.556312 (+ 5863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:58.557147 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:58.566116 (+ 8969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:58.566479 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:58.566604 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:58.568559 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:58.568564 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:58.568567 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:58.568596 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:58.570502 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:58.571108 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:58.571111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:58.571113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:58.571207 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:58.571547 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:58.571550 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:58.571552 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:58.571792 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:58.572797 (+ 1005us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:58.572899 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 05:15:58.573037 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4681,"thread_start_us":181,"threads_started":1}
Aug 11 05:15:58 dist-test-slave-04bb krb5kdc[26321](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786425358, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:15:58.942807 28246 heartbeater.cc:464] Registering TS with master...
I20260811 05:15:58.943148 28246 heartbeater.cc:511] Master 127.7.67.190:36813 requested a full tablet report, sending...
I20260811 05:15:58.944139 27919 ts_manager.cc:194] Registered new tserver with Master: 9d7a4bbb37924d83aa92aa0b3692ed1f (127.7.67.129:45913)
I20260811 05:15:58.945001 27919 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.129@KRBTEST.COM'} at 127.7.67.129:33173
I20260811 05:15:59.272336 28487 heartbeater.cc:464] Registering TS with master...
Aug 11 05:15:59 dist-test-slave-04bb krb5kdc[26321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425358, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for HTTP/127.7.67.148@KRBTEST.COM
I20260811 05:15:59.272668 28487 heartbeater.cc:511] Master 127.7.67.190:36813 requested a full tablet report, sending...
I20260811 05:15:59.275278 27918 ts_manager.cc:194] Registered new tserver with Master: df8ff797347f4ef09a5d8e7925f70c01 (127.7.67.130:44797)
I20260811 05:15:59.276105 27918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.130@KRBTEST.COM'} at 127.7.67.130:45487
I20260811 05:15:59.277105 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:15:59.293402 28744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:59.280776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42352 (local address 127.7.67.190:36813)
0811 05:15:59.281471 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:59.281476 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:59.281489 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:15:59.281528 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:59.281531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:59.281575 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:59.281655 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:15:59.282617 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.283459 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.284644 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.284979 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.285080 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:59.286834 (+ 1754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:15:59.286839 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:15:59.286842 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:15:59.286871 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:15:59.288884 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:59.289641 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:59.289647 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:59.289648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:59.289755 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:15:59.291977 (+ 2222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:15:59.291981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:15:59.291983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:15:59.292219 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:15:59.292360 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:59.293087 (+ 727us) server_negotiation.cc:299] Negotiation successful
0811 05:15:59.293218 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":89,"threads_started":1}
I20260811 05:15:59.297906 27918 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42352:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:15:59.302368 27918 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:15:59.339366 28761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:59.326018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:45731 (local address 127.0.0.1:59550)
0811 05:15:59.332833 (+ 6815us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:59.332890 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:59.333025 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:59.333756 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:59.333763 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:59.333786 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:59.333944 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:59.333952 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.335124 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.335129 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:59.337399 (+ 2270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:59.337412 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.338786 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.338790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:59.338913 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:59.338956 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 05:15:59.339067 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 05:15:59.339190 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6596,"thread_start_us":106,"threads_started":1}
I20260811 05:15:59.340811 28757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:59.326145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59550 (local address 127.7.67.131:45731)
0811 05:15:59.333327 (+ 7182us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:59.333333 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:59.333357 (+ 24us) server_negotiation.cc:407] Connection header received
0811 05:15:59.333423 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:59.333427 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:59.333593 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:59.333754 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:59.334271 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.335003 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.337551 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.338451 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.338546 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:59.338719 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:59.340555 (+ 1836us) server_negotiation.cc:299] Negotiation successful
0811 05:15:59.340669 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7064,"thread_start_us":86,"threads_started":1}
I20260811 05:15:59.344002 28758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:59.327163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:45913 (local address 127.0.0.1:59206)
0811 05:15:59.334041 (+ 6878us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:59.334056 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:59.334126 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:59.338164 (+ 4038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:59.338167 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:59.338194 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:59.338284 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:59.338291 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.342076 (+ 3785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.342080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:59.342831 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:59.342850 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.343616 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.343620 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:59.343711 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:59.343733 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 05:15:59.343798 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 05:15:59.343871 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6803,"thread_start_us":79,"threads_started":1}
I20260811 05:15:59.344058 28762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:59.327579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59206 (local address 127.7.67.129:45913)
0811 05:15:59.336208 (+ 8629us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:59.336215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:59.336240 (+ 25us) server_negotiation.cc:407] Connection header received
0811 05:15:59.336311 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:59.336318 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:59.336510 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:59.338440 (+ 1930us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:59.338595 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.339679 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.342975 (+ 3296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.343481 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.343612 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:59.343764 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:59.343821 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 05:15:59.343926 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8507,"thread_start_us":93,"threads_started":1}
I20260811 05:15:59.345249 28760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:59.326974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48194 (local address 127.7.67.130:44797)
0811 05:15:59.335308 (+ 8334us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:59.335316 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:59.335338 (+ 22us) server_negotiation.cc:407] Connection header received
0811 05:15:59.335415 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:59.335421 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:59.335616 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:59.336650 (+ 1034us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:59.336793 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.337929 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.341128 (+ 3199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.341932 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.344751 (+ 2819us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:59.344922 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:59.344990 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 05:15:59.345095 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8224,"thread_start_us":115,"threads_started":1}
I20260811 05:15:59.344767 28623 tablet_service.cc:1467] Processing CreateTablet for tablet 668ed2854b0b46d0ad910590e6a0bf8c (DEFAULT_TABLE table=test-table [id=1f41fedeac6447fc9ea5fc80578d14f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:15:59.344659 28756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:59.326813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:44797 (local address 127.0.0.1:48194)
0811 05:15:59.332832 (+ 6019us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:59.332895 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:59.333027 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:59.335779 (+ 2752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:59.335783 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:59.335814 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:59.335928 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:59.335935 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.339828 (+ 3893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.339834 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:59.340963 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:59.340975 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.344270 (+ 3295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.344274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:59.344369 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:59.344394 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 05:15:59.344444 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 05:15:59.344515 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5850,"thread_start_us":106,"threads_started":1}
I20260811 05:15:59.346104 28623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 668ed2854b0b46d0ad910590e6a0bf8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:15:59.348258 28142 tablet_service.cc:1467] Processing CreateTablet for tablet 668ed2854b0b46d0ad910590e6a0bf8c (DEFAULT_TABLE table=test-table [id=1f41fedeac6447fc9ea5fc80578d14f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:15:59.348250 28360 tablet_service.cc:1467] Processing CreateTablet for tablet 668ed2854b0b46d0ad910590e6a0bf8c (DEFAULT_TABLE table=test-table [id=1f41fedeac6447fc9ea5fc80578d14f2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:15:59.349336 28360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 668ed2854b0b46d0ad910590e6a0bf8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:15:59.349346 28142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 668ed2854b0b46d0ad910590e6a0bf8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:15:59.358513 28763 tablet_bootstrap.cc:492] T 668ed2854b0b46d0ad910590e6a0bf8c P bdb85032b7a44bfea527bc1d5d7ef7da: Bootstrap starting.
I20260811 05:15:59.358831 28767 tablet_bootstrap.cc:492] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f: Bootstrap starting.
I20260811 05:15:59.360028 28768 tablet_bootstrap.cc:492] T 668ed2854b0b46d0ad910590e6a0bf8c P df8ff797347f4ef09a5d8e7925f70c01: Bootstrap starting.
I20260811 05:15:59.361299 28763 tablet_bootstrap.cc:654] T 668ed2854b0b46d0ad910590e6a0bf8c P bdb85032b7a44bfea527bc1d5d7ef7da: Neither blocks nor log segments found. Creating new log.
I20260811 05:15:59.362229 28763 log.cc:824] T 668ed2854b0b46d0ad910590e6a0bf8c P bdb85032b7a44bfea527bc1d5d7ef7da: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:59.362344 28768 tablet_bootstrap.cc:654] T 668ed2854b0b46d0ad910590e6a0bf8c P df8ff797347f4ef09a5d8e7925f70c01: Neither blocks nor log segments found. Creating new log.
I20260811 05:15:59.363225 28768 log.cc:824] T 668ed2854b0b46d0ad910590e6a0bf8c P df8ff797347f4ef09a5d8e7925f70c01: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:59.364701 28767 tablet_bootstrap.cc:654] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f: Neither blocks nor log segments found. Creating new log.
I20260811 05:15:59.365531 28767 log.cc:824] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f: Log is configured to *not* fsync() on all Append() calls
I20260811 05:15:59.365916 28768 tablet_bootstrap.cc:492] T 668ed2854b0b46d0ad910590e6a0bf8c P df8ff797347f4ef09a5d8e7925f70c01: No bootstrap required, opened a new log
I20260811 05:15:59.366133 28768 ts_tablet_manager.cc:1397] T 668ed2854b0b46d0ad910590e6a0bf8c P df8ff797347f4ef09a5d8e7925f70c01: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:15:59.368234 28763 tablet_bootstrap.cc:492] T 668ed2854b0b46d0ad910590e6a0bf8c P bdb85032b7a44bfea527bc1d5d7ef7da: No bootstrap required, opened a new log
I20260811 05:15:59.368434 28763 ts_tablet_manager.cc:1397] T 668ed2854b0b46d0ad910590e6a0bf8c P bdb85032b7a44bfea527bc1d5d7ef7da: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260811 05:15:59.370575 28768 raft_consensus.cc:348] T 668ed2854b0b46d0ad910590e6a0bf8c P df8ff797347f4ef09a5d8e7925f70c01 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb85032b7a44bfea527bc1d5d7ef7da" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 45731 } } peers { permanent_uuid: "df8ff797347f4ef09a5d8e7925f70c01" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44797 } } peers { permanent_uuid: "9d7a4bbb37924d83aa92aa0b3692ed1f" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 45913 } }
I20260811 05:15:59.370857 28768 raft_consensus.cc:374] T 668ed2854b0b46d0ad910590e6a0bf8c P df8ff797347f4ef09a5d8e7925f70c01 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:15:59.370922 28768 raft_consensus.cc:729] T 668ed2854b0b46d0ad910590e6a0bf8c P df8ff797347f4ef09a5d8e7925f70c01 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df8ff797347f4ef09a5d8e7925f70c01, State: Initialized, Role: FOLLOWER
I20260811 05:15:59.371619 28768 consensus_queue.cc:260] T 668ed2854b0b46d0ad910590e6a0bf8c P df8ff797347f4ef09a5d8e7925f70c01 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb85032b7a44bfea527bc1d5d7ef7da" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 45731 } } peers { permanent_uuid: "df8ff797347f4ef09a5d8e7925f70c01" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44797 } } peers { permanent_uuid: "9d7a4bbb37924d83aa92aa0b3692ed1f" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 45913 } }
I20260811 05:15:59.372869 28768 ts_tablet_manager.cc:1428] T 668ed2854b0b46d0ad910590e6a0bf8c P df8ff797347f4ef09a5d8e7925f70c01: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 05:15:59.373308 28487 heartbeater.cc:503] Master 127.7.67.190:36813 was elected leader, sending a full tablet report...
I20260811 05:15:59.373142 28763 raft_consensus.cc:348] T 668ed2854b0b46d0ad910590e6a0bf8c P bdb85032b7a44bfea527bc1d5d7ef7da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb85032b7a44bfea527bc1d5d7ef7da" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 45731 } } peers { permanent_uuid: "df8ff797347f4ef09a5d8e7925f70c01" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44797 } } peers { permanent_uuid: "9d7a4bbb37924d83aa92aa0b3692ed1f" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 45913 } }
I20260811 05:15:59.373404 28763 raft_consensus.cc:374] T 668ed2854b0b46d0ad910590e6a0bf8c P bdb85032b7a44bfea527bc1d5d7ef7da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:15:59.373450 28763 raft_consensus.cc:729] T 668ed2854b0b46d0ad910590e6a0bf8c P bdb85032b7a44bfea527bc1d5d7ef7da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdb85032b7a44bfea527bc1d5d7ef7da, State: Initialized, Role: FOLLOWER
I20260811 05:15:59.373903 28763 consensus_queue.cc:260] T 668ed2854b0b46d0ad910590e6a0bf8c P bdb85032b7a44bfea527bc1d5d7ef7da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb85032b7a44bfea527bc1d5d7ef7da" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 45731 } } peers { permanent_uuid: "df8ff797347f4ef09a5d8e7925f70c01" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44797 } } peers { permanent_uuid: "9d7a4bbb37924d83aa92aa0b3692ed1f" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 45913 } }
I20260811 05:15:59.375101 28763 ts_tablet_manager.cc:1428] T 668ed2854b0b46d0ad910590e6a0bf8c P bdb85032b7a44bfea527bc1d5d7ef7da: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 05:15:59.375470 28724 heartbeater.cc:503] Master 127.7.67.190:36813 was elected leader, sending a full tablet report...
I20260811 05:15:59.376456 28767 tablet_bootstrap.cc:492] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f: No bootstrap required, opened a new log
I20260811 05:15:59.376708 28767 ts_tablet_manager.cc:1397] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.003s
I20260811 05:15:59.380916 28767 raft_consensus.cc:348] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb85032b7a44bfea527bc1d5d7ef7da" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 45731 } } peers { permanent_uuid: "df8ff797347f4ef09a5d8e7925f70c01" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44797 } } peers { permanent_uuid: "9d7a4bbb37924d83aa92aa0b3692ed1f" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 45913 } }
I20260811 05:15:59.381208 28767 raft_consensus.cc:374] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:15:59.381269 28767 raft_consensus.cc:729] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d7a4bbb37924d83aa92aa0b3692ed1f, State: Initialized, Role: FOLLOWER
I20260811 05:15:59.381770 28767 consensus_queue.cc:260] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb85032b7a44bfea527bc1d5d7ef7da" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 45731 } } peers { permanent_uuid: "df8ff797347f4ef09a5d8e7925f70c01" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44797 } } peers { permanent_uuid: "9d7a4bbb37924d83aa92aa0b3692ed1f" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 45913 } }
I20260811 05:15:59.382745 28246 heartbeater.cc:503] Master 127.7.67.190:36813 was elected leader, sending a full tablet report...
I20260811 05:15:59.382815 28767 ts_tablet_manager.cc:1428] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
W20260811 05:15:59.412022 28248 tablet.cc:2369] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:15:59.482154 28488 tablet.cc:2369] T 668ed2854b0b46d0ad910590e6a0bf8c P df8ff797347f4ef09a5d8e7925f70c01: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 05:15:59 dist-test-slave-04bb krb5kdc[26321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425358, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for HTTP/127.7.67.148@KRBTEST.COM
W20260811 05:15:59.539431 28726 tablet.cc:2369] T 668ed2854b0b46d0ad910590e6a0bf8c P bdb85032b7a44bfea527bc1d5d7ef7da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:15:59.631497 28776 raft_consensus.cc:482] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:15:59.631770 28776 raft_consensus.cc:504] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb85032b7a44bfea527bc1d5d7ef7da" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 45731 } } peers { permanent_uuid: "df8ff797347f4ef09a5d8e7925f70c01" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44797 } } peers { permanent_uuid: "9d7a4bbb37924d83aa92aa0b3692ed1f" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 45913 } }
I20260811 05:15:59.633201 28776 leader_election.cc:288] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdb85032b7a44bfea527bc1d5d7ef7da (127.7.67.131:45731), df8ff797347f4ef09a5d8e7925f70c01 (127.7.67.130:44797)
I20260811 05:15:59.641065 28757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:59.634779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:45735 (local address 127.7.67.131:45731)
0811 05:15:59.634908 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:59.634914 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:59.636025 (+ 1111us) server_negotiation.cc:407] Connection header received
0811 05:15:59.636252 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:59.636256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:59.636312 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:59.636360 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:59.636969 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.637782 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.639602 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.640235 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.640335 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:59.640391 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:59.640843 (+ 452us) server_negotiation.cc:299] Negotiation successful
0811 05:15:59.640930 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 05:15:59.641772 28779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:59.633646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:45731 (local address 127.7.67.129:45735)
0811 05:15:59.635909 (+ 2263us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:59.635926 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:59.636120 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:59.636504 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:59.636507 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:59.636551 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:59.636820 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:59.636828 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.637940 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.637943 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:59.639444 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:59.639457 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.640496 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.640499 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:59.640670 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:59.640697 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 05:15:59.641519 (+ 822us) client_negotiation.cc:240] Negotiation successful
0811 05:15:59.641609 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2184,"thread_start_us":178,"threads_started":1}
I20260811 05:15:59.642750 28650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668ed2854b0b46d0ad910590e6a0bf8c" candidate_uuid: "9d7a4bbb37924d83aa92aa0b3692ed1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdb85032b7a44bfea527bc1d5d7ef7da" is_pre_election: true
I20260811 05:15:59.643107 28650 raft_consensus.cc:2454] T 668ed2854b0b46d0ad910590e6a0bf8c P bdb85032b7a44bfea527bc1d5d7ef7da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d7a4bbb37924d83aa92aa0b3692ed1f in term 0.
I20260811 05:15:59.643805 28063 leader_election.cc:302] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d7a4bbb37924d83aa92aa0b3692ed1f, bdb85032b7a44bfea527bc1d5d7ef7da; no voters:
I20260811 05:15:59.644135 28776 raft_consensus.cc:2781] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:15:59.644251 28776 raft_consensus.cc:482] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:15:59.644302 28776 raft_consensus.cc:3037] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:15:59.646054 28776 raft_consensus.cc:504] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb85032b7a44bfea527bc1d5d7ef7da" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 45731 } } peers { permanent_uuid: "df8ff797347f4ef09a5d8e7925f70c01" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44797 } } peers { permanent_uuid: "9d7a4bbb37924d83aa92aa0b3692ed1f" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 45913 } }
I20260811 05:15:59.646433 28776 leader_election.cc:288] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f [CANDIDATE]: Term 1 election: Requested vote from peers bdb85032b7a44bfea527bc1d5d7ef7da (127.7.67.131:45731), df8ff797347f4ef09a5d8e7925f70c01 (127.7.67.130:44797)
I20260811 05:15:59.647354 28650 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668ed2854b0b46d0ad910590e6a0bf8c" candidate_uuid: "9d7a4bbb37924d83aa92aa0b3692ed1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdb85032b7a44bfea527bc1d5d7ef7da"
I20260811 05:15:59.647487 28650 raft_consensus.cc:3037] T 668ed2854b0b46d0ad910590e6a0bf8c P bdb85032b7a44bfea527bc1d5d7ef7da [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:15:59.648986 28650 raft_consensus.cc:2454] T 668ed2854b0b46d0ad910590e6a0bf8c P bdb85032b7a44bfea527bc1d5d7ef7da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9d7a4bbb37924d83aa92aa0b3692ed1f in term 1.
I20260811 05:15:59.649475 28063 leader_election.cc:302] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9d7a4bbb37924d83aa92aa0b3692ed1f, bdb85032b7a44bfea527bc1d5d7ef7da; no voters:
I20260811 05:15:59.649693 28776 raft_consensus.cc:2781] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:15:59.649953 28776 raft_consensus.cc:686] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f [term 1 LEADER]: Becoming Leader. State: Replica: 9d7a4bbb37924d83aa92aa0b3692ed1f, State: Running, Role: LEADER
I20260811 05:15:59.650316 28776 consensus_queue.cc:237] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb85032b7a44bfea527bc1d5d7ef7da" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 45731 } } peers { permanent_uuid: "df8ff797347f4ef09a5d8e7925f70c01" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44797 } } peers { permanent_uuid: "9d7a4bbb37924d83aa92aa0b3692ed1f" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 45913 } }
I20260811 05:15:59.653903 27919 catalog_manager.cc:5495] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f reported cstate change: term changed from 0 to 1, leader changed from <none> to 9d7a4bbb37924d83aa92aa0b3692ed1f (127.7.67.129). New cstate: current_term: 1 leader_uuid: "9d7a4bbb37924d83aa92aa0b3692ed1f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdb85032b7a44bfea527bc1d5d7ef7da" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 45731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df8ff797347f4ef09a5d8e7925f70c01" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d7a4bbb37924d83aa92aa0b3692ed1f" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 45913 } health_report { overall_health: HEALTHY } } }
I20260811 05:15:59.666599 28760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:59.634937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:43417 (local address 127.7.67.130:44797)
0811 05:15:59.635048 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:59.635053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:59.641681 (+ 6628us) server_negotiation.cc:407] Connection header received
0811 05:15:59.646590 (+ 4909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:59.646595 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:59.646655 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:59.646740 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:15:59.659724 (+ 12984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.660632 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.663960 (+ 3328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.664631 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.664734 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:59.664791 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:59.666281 (+ 1490us) server_negotiation.cc:299] Negotiation successful
0811 05:15:59.666451 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 05:15:59.667358 28780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:59.634191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:44797 (local address 127.7.67.129:43417)
0811 05:15:59.641549 (+ 7358us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:59.641567 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 05:15:59.644655 (+ 3088us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:15:59.659018 (+ 14363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:15:59.659023 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:15:59.659049 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:15:59.659255 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:59.659264 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.660789 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.660792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:59.662170 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:15:59.662182 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.665950 (+ 3768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.665959 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:15:59.666115 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:59.666145 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 05:15:59.667049 (+ 904us) client_negotiation.cc:240] Negotiation successful
0811 05:15:59.667145 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7282,"thread_start_us":132,"threads_started":1}
I20260811 05:15:59.668349 28373 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668ed2854b0b46d0ad910590e6a0bf8c" candidate_uuid: "9d7a4bbb37924d83aa92aa0b3692ed1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df8ff797347f4ef09a5d8e7925f70c01" is_pre_election: true
I20260811 05:15:59.668377 28402 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "668ed2854b0b46d0ad910590e6a0bf8c" candidate_uuid: "9d7a4bbb37924d83aa92aa0b3692ed1f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df8ff797347f4ef09a5d8e7925f70c01"
I20260811 05:15:59.668743 28373 raft_consensus.cc:2454] T 668ed2854b0b46d0ad910590e6a0bf8c P df8ff797347f4ef09a5d8e7925f70c01 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9d7a4bbb37924d83aa92aa0b3692ed1f in term 0.
I20260811 05:15:59.704808 28762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:59.698944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59214 (local address 127.7.67.129:45913)
0811 05:15:59.699074 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 05:15:59.699078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:15:59.699092 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:15:59.699184 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:15:59.699188 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:15:59.699262 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:15:59.699396 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:15:59.700042 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.700914 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.702181 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:15:59.702526 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:15:59.702629 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:15:59.703127 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:15:59.703746 (+ 619us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:15:59.703862 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 05:15:59.704440 (+ 578us) server_negotiation.cc:299] Negotiation successful
0811 05:15:59.704510 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 05:15:59.725914 28373 raft_consensus.cc:3037] T 668ed2854b0b46d0ad910590e6a0bf8c P df8ff797347f4ef09a5d8e7925f70c01 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:15:59.728024 28373 raft_consensus.cc:1261] T 668ed2854b0b46d0ad910590e6a0bf8c P df8ff797347f4ef09a5d8e7925f70c01 [term 1 FOLLOWER]: Refusing update from remote peer 9d7a4bbb37924d83aa92aa0b3692ed1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:15:59.728981 28776 consensus_queue.cc:1035] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "df8ff797347f4ef09a5d8e7925f70c01" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:15:59.730585 28650 raft_consensus.cc:1261] T 668ed2854b0b46d0ad910590e6a0bf8c P bdb85032b7a44bfea527bc1d5d7ef7da [term 1 FOLLOWER]: Refusing update from remote peer 9d7a4bbb37924d83aa92aa0b3692ed1f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:15:59.731508 28782 consensus_queue.cc:1035] T 668ed2854b0b46d0ad910590e6a0bf8c P 9d7a4bbb37924d83aa92aa0b3692ed1f [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdb85032b7a44bfea527bc1d5d7ef7da" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 45731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:15:59.751914 28792 mvcc.cc:205] Tried to move back new op lower bound from 7317198273425805312 to 7317198273131495424. Current Snapshot: MvccSnapshot[applied={T|T < 7317198273425805312}]
I20260811 05:15:59.762387 27919 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:15:59.762602 27919 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42352:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:15:59.765883 27919 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 54c062c181e841438dd2dd42a41eb06b: Sending DeleteTablet for 3 replicas of tablet 668ed2854b0b46d0ad910590e6a0bf8c
I20260811 05:15:59.767481 28623 tablet_service.cc:1514] Processing DeleteTablet for tablet 668ed2854b0b46d0ad910590e6a0bf8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:15:59 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:59550
I20260811 05:15:59.768152 28360 tablet_service.cc:1514] Processing DeleteTablet for tablet 668ed2854b0b46d0ad910590e6a0bf8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:15:59 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:48194
I20260811 05:15:59.768903 28142 tablet_service.cc:1514] Processing DeleteTablet for tablet 668ed2854b0b46d0ad910590e6a0bf8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:15:59 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:59206
I20260811 05:15:59.770321 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 27982
I20260811 05:15:59.785709 28813 tablet_replica.cc:330] T 668ed2854b0b46d0ad910590e6a0bf8c P df8ff797347f4ef09a5d8e7925f70c01: stopping tablet replica
I20260811 05:15:59.785955 28810 tablet_replica.cc:330] T 668ed2854b0b46d0ad910590e6a0bf8c P bdb85032b7a44bfea527bc1d5d7ef7da: stopping tablet replica
I20260811 05:15:59.786810 28810 raft_consensus.cc:2229] T 668ed2854b0b46d0ad910590e6a0bf8c P bdb85032b7a44bfea527bc1d5d7ef7da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:15:59.787026 28813 raft_consensus.cc:2229] T 668ed2854b0b46d0ad910590e6a0bf8c P df8ff797347f4ef09a5d8e7925f70c01 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:15:59.787464 28810 raft_consensus.cc:2258] T 668ed2854b0b46d0ad910590e6a0bf8c P bdb85032b7a44bfea527bc1d5d7ef7da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:15:59.787859 28813 raft_consensus.cc:2258] T 668ed2854b0b46d0ad910590e6a0bf8c P df8ff797347f4ef09a5d8e7925f70c01 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:15:59.794173 28813 ts_tablet_manager.cc:1905] T 668ed2854b0b46d0ad910590e6a0bf8c P df8ff797347f4ef09a5d8e7925f70c01: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:15:59.797832 28813 ts_tablet_manager.cc:1918] T 668ed2854b0b46d0ad910590e6a0bf8c P df8ff797347f4ef09a5d8e7925f70c01: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:15:59.797952 28813 log.cc:1196] T 668ed2854b0b46d0ad910590e6a0bf8c P df8ff797347f4ef09a5d8e7925f70c01: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/668ed2854b0b46d0ad910590e6a0bf8c
I20260811 05:15:59.798364 28813 ts_tablet_manager.cc:1939] T 668ed2854b0b46d0ad910590e6a0bf8c P df8ff797347f4ef09a5d8e7925f70c01: Deleting consensus metadata
W20260811 05:15:59.798509 27901 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.129:45913 (error 108)
I20260811 05:15:59.800091 27901 catalog_manager.cc:4841] TS df8ff797347f4ef09a5d8e7925f70c01 (127.7.67.130:44797): tablet 668ed2854b0b46d0ad910590e6a0bf8c (table test-table [id=1f41fedeac6447fc9ea5fc80578d14f2]) successfully deleted
I20260811 05:15:59.800877 28758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:15:59.799934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:45913 (local address 127.0.0.1:59216)
0811 05:15:59.800359 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0811 05:15:59.800434 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.129:45913: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329}
W20260811 05:15:59.801165 27901 catalog_manager.cc:4584] TS 9d7a4bbb37924d83aa92aa0b3692ed1f (127.7.67.129:45913): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 668ed2854b0b46d0ad910590e6a0bf8c: Network error: Client connection negotiation failed: client connection to 127.7.67.129:45913: connect: Connection refused (error 111)
I20260811 05:15:59.801599 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 28254
I20260811 05:15:59.808120 28810 ts_tablet_manager.cc:1905] T 668ed2854b0b46d0ad910590e6a0bf8c P bdb85032b7a44bfea527bc1d5d7ef7da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:15:59.812057 28810 ts_tablet_manager.cc:1918] T 668ed2854b0b46d0ad910590e6a0bf8c P bdb85032b7a44bfea527bc1d5d7ef7da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:15:59.812187 28810 log.cc:1196] T 668ed2854b0b46d0ad910590e6a0bf8c P bdb85032b7a44bfea527bc1d5d7ef7da: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/668ed2854b0b46d0ad910590e6a0bf8c
I20260811 05:15:59.812604 28810 ts_tablet_manager.cc:1939] T 668ed2854b0b46d0ad910590e6a0bf8c P bdb85032b7a44bfea527bc1d5d7ef7da: Deleting consensus metadata
I20260811 05:15:59.814157 27901 catalog_manager.cc:4841] TS bdb85032b7a44bfea527bc1d5d7ef7da (127.7.67.131:45731): tablet 668ed2854b0b46d0ad910590e6a0bf8c (table test-table [id=1f41fedeac6447fc9ea5fc80578d14f2]) successfully deleted
I20260811 05:15:59.818671 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 28490
I20260811 05:15:59.833168 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 27865
I20260811 05:15:59.855162 7438 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 05:16:00.201385 7438 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 05:16:00.201506 7438 mini_ranger.cc:66] Stopping Ranger...
I20260811 05:16:00.557960 7438 mini_ranger.cc:68] Stopped Ranger
2026-08-11 05:16:00.558 UTC [26371] LOG: received smart shutdown request
2026-08-11 05:16:00.563 UTC [26371] LOG: background worker "logical replication launcher" (PID 26384) exited with exit code 1
2026-08-11 05:16:00.564 UTC [26377] LOG: shutting down
2026-08-11 05:16:00.752 UTC [26371] LOG: database system is shut down
2026-08-11T05:16:00Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51695 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:16:00 dist-test-slave-04bb krb5kdc[28832](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:16:00 dist-test-slave-04bb krb5kdc[28832](info): set up 2 sockets
Aug 11 05:16:00 dist-test-slave-04bb krb5kdc[28832](info): commencing operation
krb5kdc: starting...
W20260811 05:16:02.165335 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:16:02 dist-test-slave-04bb krb5kdc[28832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:16:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:16:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:16:02.299675 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:39699
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:33341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:39699,127.7.67.189:41025,127.7.67.188:45433
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:16:02.418143 28851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:02.418469 28851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:02.418540 28851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:02.422636 28851 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:16:02.422724 28851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:02.422763 28851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:02.422799 28851 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:16:02.422837 28851 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:16:02.428234 28851 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:33341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:39699,127.7.67.189:41025,127.7.67.188:45433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:39699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:16:02.429461 28851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:02.430557 28851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:02.439746 28860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:02.439751 28862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:02.439751 28864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:02.440090 28851 server_base.cc:1034] running on GCE node
I20260811 05:16:02.440834 28851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:02.442078 28851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:02.443269 28851 hybrid_clock.cc:648] HybridClock initialized: now 1786425362443238 us; error 47 us; skew 500 ppm
Aug 11 05:16:02 dist-test-slave-04bb krb5kdc[28832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425362, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:02.446491 28851 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:16:02.447500 28851 webserver.cc:460] Webserver started at http://127.7.67.190:36529/ using document root <none> and password file <none>
I20260811 05:16:02.448179 28851 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:02.448275 28851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:02.448559 28851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:16:02.450709 28851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "36b73c3b875046b48df6053302ccd4e7"
format_stamp: "Formatted at 2026-08-11 05:16:02 on dist-test-slave-04bb"
server_key: "366afb0264412fe6cad40b98544b5456"
server_key_iv: "d08af8e65f6a142186cb6db4e592b883"
server_key_version: "encryptionkey@0"
I20260811 05:16:02.451365 28851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "36b73c3b875046b48df6053302ccd4e7"
format_stamp: "Formatted at 2026-08-11 05:16:02 on dist-test-slave-04bb"
server_key: "366afb0264412fe6cad40b98544b5456"
server_key_iv: "d08af8e65f6a142186cb6db4e592b883"
server_key_version: "encryptionkey@0"
I20260811 05:16:02.455879 28851 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 05:16:02.458503 28875 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:02.459782 28851 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 05:16:02.459950 28851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "36b73c3b875046b48df6053302ccd4e7"
format_stamp: "Formatted at 2026-08-11 05:16:02 on dist-test-slave-04bb"
server_key: "366afb0264412fe6cad40b98544b5456"
server_key_iv: "d08af8e65f6a142186cb6db4e592b883"
server_key_version: "encryptionkey@0"
I20260811 05:16:02.460069 28851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:02.482584 28851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:02.486088 28851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:02.486310 28851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:02.493888 28851 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:39699
I20260811 05:16:02.493968 28960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:39699 every 8 connection(s)
I20260811 05:16:02.494913 28851 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:16:02.497287 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 28851
I20260811 05:16:02.497413 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:16:02.497741 7438 external_mini_cluster.cc:1428] Setting key 1c40d1284e6b05cce0fe21b27e617e7c
I20260811 05:16:02.498286 28961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:02.502737 28961 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39699 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.67.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.189@KRBTEST.COM" created.
Aug 11 05:16:02 dist-test-slave-04bb krb5kdc[28832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425362, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:16:02.519179 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.505204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:39699 (local address 127.0.0.1:58792)
0811 05:16:02.505822 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.505858 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:02.506758 (+ 900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:02.507456 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:02.507461 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:02.507901 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:02.508169 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:02.508183 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.509420 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.509423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:02.510206 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:02.510215 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.510910 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.510917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:02.511083 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:02.511760 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:02.511787 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:02.513410 (+ 1623us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:02.515916 (+ 2506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:02.515925 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:02.515934 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:02.516208 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:02.516678 (+ 470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:02.516680 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:02.516681 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:02.516797 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:02.517644 (+ 847us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:02.517650 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:02.517792 (+ 142us) client_negotiation.cc:769] Sending connection context
0811 05:16:02.517942 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 05:16:02.518241 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"spinlock_wait_cycles":17152,"thread_start_us":214,"threads_started":1}
I20260811 05:16:02.519184 28964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.505204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58792 (local address 127.7.67.190:39699)
0811 05:16:02.505673 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:02.505694 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:02.505966 (+ 272us) server_negotiation.cc:407] Connection header received
0811 05:16:02.507068 (+ 1102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:02.507074 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:02.507267 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:02.507457 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:02.508317 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.509267 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.510346 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.510730 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.510860 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:02.513581 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:02.513591 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:02.513609 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:02.513643 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:02.515787 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:02.516345 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:02.516354 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:02.516359 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:02.516514 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:02.516903 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:02.516909 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:02.516911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:02.517507 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:02.517667 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:02.518069 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 05:16:02.518385 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":85,"server-negotiator.queue_time_us":309,"thread_start_us":160,"threads_started":1}
I20260811 05:16:02.521849 28961 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:16:02.522620 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.522355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:41025 (local address 127.0.0.1:41558)
0811 05:16:02.522488 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.522560 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.189:41025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 05:16:02.523016 28881 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.67.189:41025: connect: Connection refused (error 111)
I20260811 05:16:02.523849 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.523651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:41025 (local address 127.0.0.1:41566)
0811 05:16:02.523745 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.523797 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.189:41025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 05:16:02.524089 28961 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.189:41025: Network error: Client connection negotiation failed: client connection to 127.7.67.189:41025: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.189.keytab.
Entry for principal kudu/127.7.67.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.189.keytab.
I20260811 05:16:02.527674 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.189:41025
--webserver_interface=127.7.67.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.189.keytab
--principal=kudu/127.7.67.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:33341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:39699,127.7.67.189:41025,127.7.67.188:45433
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:16:02.573359 28961 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } attempt: 1
I20260811 05:16:02.574321 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.573941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:41025 (local address 127.0.0.1:41574)
0811 05:16:02.574137 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.574222 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.189:41025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 05:16:02.575115 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.574890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:41025 (local address 127.0.0.1:41586)
0811 05:16:02.574997 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.575052 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.189:41025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 05:16:02.575330 28961 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.189:41025: Network error: Client connection negotiation failed: client connection to 127.7.67.189:41025: connect: Connection refused (error 111)
I20260811 05:16:02.643563 28961 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } attempt: 2
I20260811 05:16:02.644621 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.644210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:41025 (local address 127.0.0.1:41596)
0811 05:16:02.644412 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.644492 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.189:41025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 05:16:02.645466 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.645233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:41025 (local address 127.0.0.1:41610)
0811 05:16:02.645351 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.645405 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.189:41025: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 05:16:02.645625 28961 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.189:41025: Network error: Client connection negotiation failed: client connection to 127.7.67.189:41025: connect: Connection refused (error 111)
W20260811 05:16:02.646847 28970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:02.647105 28970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:02.647177 28970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:02.652086 28970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:16:02.652176 28970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:02.652201 28970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:02.652220 28970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:16:02.652288 28970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:16:02.658778 28970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:33341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:39699,127.7.67.189:41025,127.7.67.188:45433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.189.keytab
--principal=kudu/127.7.67.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.189:41025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.67.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:16:02.660082 28970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:02.661345 28970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:02.670970 28978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:02.670982 28981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:02.670986 28979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:02.671315 28970 server_base.cc:1034] running on GCE node
I20260811 05:16:02.671936 28970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:02.673189 28970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:02.674386 28970 hybrid_clock.cc:648] HybridClock initialized: now 1786425362674369 us; error 33 us; skew 500 ppm
Aug 11 05:16:02 dist-test-slave-04bb krb5kdc[28832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425362, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:02.677284 28970 init.cc:377] Logged in from keytab as kudu/127.7.67.189@KRBTEST.COM (short username kudu)
I20260811 05:16:02.678150 28970 webserver.cc:460] Webserver started at http://127.7.67.189:37389/ using document root <none> and password file <none>
I20260811 05:16:02.678681 28970 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:02.678740 28970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:02.678928 28970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:16:02.681064 28970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-1/data/instance:
uuid: "8af95784b7e84f459ca1daf81e7bcc26"
format_stamp: "Formatted at 2026-08-11 05:16:02 on dist-test-slave-04bb"
server_key: "9cd7b31e422aac1af53a6cee19164e9e"
server_key_iv: "97b419a9fa928ff19f062b70495ccf1b"
server_key_version: "encryptionkey@0"
I20260811 05:16:02.681849 28970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-1/wal/instance:
uuid: "8af95784b7e84f459ca1daf81e7bcc26"
format_stamp: "Formatted at 2026-08-11 05:16:02 on dist-test-slave-04bb"
server_key: "9cd7b31e422aac1af53a6cee19164e9e"
server_key_iv: "97b419a9fa928ff19f062b70495ccf1b"
server_key_version: "encryptionkey@0"
I20260811 05:16:02.687433 28970 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 05:16:02.690848 28991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:02.692163 28970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 05:16:02.692338 28970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-1/wal
uuid: "8af95784b7e84f459ca1daf81e7bcc26"
format_stamp: "Formatted at 2026-08-11 05:16:02 on dist-test-slave-04bb"
server_key: "9cd7b31e422aac1af53a6cee19164e9e"
server_key_iv: "97b419a9fa928ff19f062b70495ccf1b"
server_key_version: "encryptionkey@0"
I20260811 05:16:02.692453 28970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:02.711692 28970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:02.715101 28970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:02.715327 28970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:02.723130 28970 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.189:41025
I20260811 05:16:02.723196 29071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.189:41025 every 8 connection(s)
I20260811 05:16:02.724181 28970 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-1/data/info.pb
I20260811 05:16:02.725036 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 28970
I20260811 05:16:02.725180 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-1/wal/instance
I20260811 05:16:02.725466 7438 external_mini_cluster.cc:1428] Setting key b6fd993468008630df1046c4333c64b4
I20260811 05:16:02.727448 29073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:02.731775 29073 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39699 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.67.188@KRBTEST.COM; defaulting to no policy
I20260811 05:16:02.736773 28961 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.188@KRBTEST.COM" created.
Aug 11 05:16:02 dist-test-slave-04bb krb5kdc[28832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425362, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.189@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
Aug 11 05:16:02 dist-test-slave-04bb krb5kdc[28832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425362, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for kudu/127.7.67.189@KRBTEST.COM
I20260811 05:16:02.748005 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.737269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:41025 (local address 127.0.0.1:41624)
0811 05:16:02.737463 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.737478 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:02.737559 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:02.738254 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:02.738260 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:02.738473 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:02.739119 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:02.739134 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.740357 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.740361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:02.740931 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:02.740940 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.741700 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.741703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:02.741813 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:02.742288 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:02.742300 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:02.743932 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:02.746181 (+ 2249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:02.746184 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:02.746186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:02.746391 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:02.746734 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:02.746737 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:02.746739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:02.746827 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:02.747445 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:02.747448 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:02.747550 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 05:16:02.747668 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 05:16:02.747813 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124}
I20260811 05:16:02.748340 28964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.733801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58800 (local address 127.7.67.190:39699)
0811 05:16:02.733974 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:02.733979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:02.734866 (+ 887us) server_negotiation.cc:407] Connection header received
0811 05:16:02.735984 (+ 1118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:02.735988 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:02.736050 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:02.736143 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:02.737734 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.738511 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.739828 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.740233 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.740356 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:02.743808 (+ 3452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:02.743815 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:02.743822 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:02.743858 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:02.745907 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:02.746545 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:02.746551 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:02.746554 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:02.746647 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:02.747109 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:02.747115 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:02.747118 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:02.747389 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:02.747465 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:02.748059 (+ 594us) server_negotiation.cc:299] Negotiation successful
0811 05:16:02.748186 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 05:16:02.749260 29081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.737454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41624 (local address 127.7.67.189:41025)
0811 05:16:02.737797 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:02.737809 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:02.737831 (+ 22us) server_negotiation.cc:407] Connection header received
0811 05:16:02.737871 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:02.737876 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:02.738067 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:02.738228 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:02.739297 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.740240 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.741049 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.741574 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.741698 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:02.744071 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:02.744078 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:02.744091 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:02.744121 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:02.746059 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:02.746512 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:02.746518 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:02.746520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:02.746635 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:02.746927 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:02.746931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:02.746933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:02.747327 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:02.747483 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:02.747795 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 05:16:02.748152 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":239,"thread_start_us":121,"threads_started":1}
I20260811 05:16:02.749291 29078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.734178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:39699 (local address 127.0.0.1:58800)
0811 05:16:02.734687 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.734750 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:02.735761 (+ 1011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:02.736345 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:02.736355 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:02.736803 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:02.737560 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:02.737583 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.738634 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.738638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:02.739698 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:02.739709 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.740606 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.740614 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:02.740875 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:02.741606 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:02.741633 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:02.743591 (+ 1958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:02.746056 (+ 2465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:02.746066 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:02.746079 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:02.746416 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:02.746803 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:02.746806 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:02.746808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:02.746973 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:02.747557 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:02.747564 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:02.747758 (+ 194us) client_negotiation.cc:769] Sending connection context
0811 05:16:02.747980 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 05:16:02.748216 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":30,"thread_start_us":150,"threads_started":1}
I20260811 05:16:02.751274 29073 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:16:02.751488 28961 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:16:02.752170 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.751928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:45433 (local address 127.0.0.1:41728)
0811 05:16:02.752033 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.752095 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:45433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 05:16:02.752951 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.752706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:45433 (local address 127.0.0.1:41736)
0811 05:16:02.752849 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.752901 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:45433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260811 05:16:02.753151 28961 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.188:45433: Network error: Client connection negotiation failed: client connection to 127.7.67.188:45433: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.188.keytab.
Entry for principal kudu/127.7.67.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.188.keytab.
I20260811 05:16:02.755954 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.188:45433
--webserver_interface=127.7.67.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.188.keytab
--principal=kudu/127.7.67.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:33341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:39699,127.7.67.189:41025,127.7.67.188:45433
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:16:02 dist-test-slave-04bb krb5kdc[28832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425362, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.189@KRBTEST.COM for kudu/127.7.67.189@KRBTEST.COM
I20260811 05:16:02.760615 29078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.751859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:41025 (local address 127.0.0.1:41640)
0811 05:16:02.752300 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.752317 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:02.752415 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:02.752833 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:02.752835 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:02.752981 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:02.753098 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:02.753106 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.753924 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.753926 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:02.754586 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:02.754596 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.755078 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.755081 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:02.755194 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:02.755731 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:02.755743 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:02.757103 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:02.759118 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:02.759121 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:02.759122 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:02.759314 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:02.759624 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:02.759626 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:02.759627 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:02.759700 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:02.760160 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:02.760162 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:02.760240 (+ 78us) client_negotiation.cc:769] Sending connection context
0811 05:16:02.760315 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 05:16:02.760438 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":1}
I20260811 05:16:02.760846 29081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.751908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41640 (local address 127.7.67.189:41025)
0811 05:16:02.752040 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:02.752045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:02.752367 (+ 322us) server_negotiation.cc:407] Connection header received
0811 05:16:02.752508 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:02.752512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:02.752611 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:02.752703 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:02.753236 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.753828 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.754728 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.754963 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.755087 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:02.757262 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:02.757266 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:02.757270 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:02.757298 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:02.758998 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:02.759420 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:02.759426 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:02.759431 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:02.759527 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:02.759786 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:02.759792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:02.759797 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:02.760057 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:02.760162 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:02.760399 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 05:16:02.760576 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":51}
I20260811 05:16:02.761771 29073 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:16:02.762533 29078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.762266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:45433 (local address 127.0.0.1:41750)
0811 05:16:02.762402 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.762482 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:45433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 05:16:02.763384 28994 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.67.188:45433: connect: Connection refused (error 111)
I20260811 05:16:02.764461 29078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.764208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:45433 (local address 127.0.0.1:41758)
0811 05:16:02.764311 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.764394 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:45433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 05:16:02.764693 29073 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.188:45433: Network error: Client connection negotiation failed: client connection to 127.7.67.188:45433: connect: Connection refused (error 111)
I20260811 05:16:02.784375 28961 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } attempt: 1
I20260811 05:16:02.785367 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.785012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:45433 (local address 127.0.0.1:41760)
0811 05:16:02.785189 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.785266 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:45433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 05:16:02.786190 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.785946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:45433 (local address 127.0.0.1:41768)
0811 05:16:02.786054 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.786107 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:45433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 05:16:02.786360 28961 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.188:45433: Network error: Client connection negotiation failed: client connection to 127.7.67.188:45433: connect: Connection refused (error 111)
I20260811 05:16:02.813969 29073 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } attempt: 1
I20260811 05:16:02.814880 29078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.814576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:45433 (local address 127.0.0.1:41780)
0811 05:16:02.814731 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.814797 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:45433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 05:16:02.815626 29078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.815405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:45433 (local address 127.0.0.1:41788)
0811 05:16:02.815530 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.815574 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:45433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 05:16:02.815769 29073 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.188:45433: Network error: Client connection negotiation failed: client connection to 127.7.67.188:45433: connect: Connection refused (error 111)
I20260811 05:16:02.861532 28961 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } attempt: 2
I20260811 05:16:02.862479 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.862116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:45433 (local address 127.0.0.1:41798)
0811 05:16:02.862293 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.862371 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:45433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 05:16:02.863302 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.863051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:45433 (local address 127.0.0.1:41812)
0811 05:16:02.863169 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.863224 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:45433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 05:16:02.863462 28961 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.188:45433: Network error: Client connection negotiation failed: client connection to 127.7.67.188:45433: connect: Connection refused (error 111)
W20260811 05:16:02.877292 29084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:02.877560 29084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:02.877601 29084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:02.881629 29084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:16:02.881696 29084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:02.881716 29084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:02.881733 29084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:16:02.881749 29084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:16:02.883941 29073 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } attempt: 2
I20260811 05:16:02.884871 29078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.884559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:45433 (local address 127.0.0.1:41822)
0811 05:16:02.884721 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.884788 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:45433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 05:16:02.885617 29078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.885425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:45433 (local address 127.0.0.1:41834)
0811 05:16:02.885520 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.885565 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:45433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 05:16:02.885802 29073 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.188:45433: Network error: Client connection negotiation failed: client connection to 127.7.67.188:45433: connect: Connection refused (error 111)
I20260811 05:16:02.887660 29084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:33341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:39699,127.7.67.189:41025,127.7.67.188:45433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.188.keytab
--principal=kudu/127.7.67.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.188:45433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.67.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:16:02.888877 29084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:02.889935 29084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:02.898908 29095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:02.898897 29097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:02.898976 29084 server_base.cc:1034] running on GCE node
W20260811 05:16:02.898897 29099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:02.899884 29084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:02.901122 29084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:02.902312 29084 hybrid_clock.cc:648] HybridClock initialized: now 1786425362902293 us; error 33 us; skew 500 ppm
Aug 11 05:16:02 dist-test-slave-04bb krb5kdc[28832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425362, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:02.905123 29084 init.cc:377] Logged in from keytab as kudu/127.7.67.188@KRBTEST.COM (short username kudu)
I20260811 05:16:02.906080 29084 webserver.cc:460] Webserver started at http://127.7.67.188:34909/ using document root <none> and password file <none>
I20260811 05:16:02.906636 29084 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:02.906695 29084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:02.906870 29084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:16:02.909050 29084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-2/data/instance:
uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a"
format_stamp: "Formatted at 2026-08-11 05:16:02 on dist-test-slave-04bb"
server_key: "dc116054d086772dc1e4aa6fa2a72117"
server_key_iv: "ab4db88426b1226dd1dd50b644ae4b25"
server_key_version: "encryptionkey@0"
I20260811 05:16:02.909582 29084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-2/wal/instance:
uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a"
format_stamp: "Formatted at 2026-08-11 05:16:02 on dist-test-slave-04bb"
server_key: "dc116054d086772dc1e4aa6fa2a72117"
server_key_iv: "ab4db88426b1226dd1dd50b644ae4b25"
server_key_version: "encryptionkey@0"
I20260811 05:16:02.913842 29084 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 05:16:02.916949 29115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:02.918227 29084 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:16:02.918359 29084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-2/wal
uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a"
format_stamp: "Formatted at 2026-08-11 05:16:02 on dist-test-slave-04bb"
server_key: "dc116054d086772dc1e4aa6fa2a72117"
server_key_iv: "ab4db88426b1226dd1dd50b644ae4b25"
server_key_version: "encryptionkey@0"
I20260811 05:16:02.918478 29084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:02.930475 29084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:02.933658 29084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:02.933902 29084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:02.941617 29084 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.188:45433
I20260811 05:16:02.941632 29192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.188:45433 every 8 connection(s)
I20260811 05:16:02.942672 29084 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-2/data/info.pb
I20260811 05:16:02.944365 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 29084
I20260811 05:16:02.944507 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786425190664461-7438-0/minicluster-data/master-2/wal/instance
I20260811 05:16:02.944875 7438 external_mini_cluster.cc:1428] Setting key f63b4a7efaac5d07ebce8045888d0b3d
I20260811 05:16:02.946126 29193 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:02.950359 29193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39699 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:16:02 dist-test-slave-04bb krb5kdc[28832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.188@KRBTEST.COM
Aug 11 05:16:02 dist-test-slave-04bb krb5kdc[28832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425362, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.188@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:16:02.960201 29196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.946394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41844 (local address 127.7.67.188:45433)
0811 05:16:02.946834 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:02.946852 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:02.946889 (+ 37us) server_negotiation.cc:407] Connection header received
0811 05:16:02.947542 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:02.947559 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:02.947916 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:02.948259 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:02.949163 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.950188 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.951115 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.951594 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.951779 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:02.954882 (+ 3103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:02.954894 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:02.954904 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:02.954931 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:02.956841 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:02.957313 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:02.957318 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:02.957322 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:02.957434 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:02.957767 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:02.957769 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:02.957770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:02.958222 (+ 452us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:02.958520 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:02.958795 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 05:16:02.959103 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":146,"threads_started":1}
I20260811 05:16:02.962816 28964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.952319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58810 (local address 127.7.67.190:39699)
0811 05:16:02.952450 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:02.952455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:02.952887 (+ 432us) server_negotiation.cc:407] Connection header received
0811 05:16:02.953057 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:02.953061 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:02.953132 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:02.953236 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:02.954199 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.954734 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.955822 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.956164 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.956288 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:02.959086 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:02.959089 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:02.959092 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:02.959120 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:02.960809 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:02.961333 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:02.961342 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:02.961343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:02.961428 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:02.961738 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:02.961744 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:02.961745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:02.961951 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:02.962072 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:02.962479 (+ 407us) server_negotiation.cc:299] Negotiation successful
0811 05:16:02.962611 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 05:16:02.962813 29197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.952230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:39699 (local address 127.0.0.1:58810)
0811 05:16:02.952729 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.952786 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:02.952924 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:02.953335 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:02.953341 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:02.953834 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:02.954054 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:02.954063 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.954863 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.954865 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:02.955684 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:02.955695 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.956305 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.956308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:02.956405 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:02.957347 (+ 942us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:02.957362 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:02.958928 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:02.960936 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:02.960945 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:02.960949 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:02.961207 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:02.961537 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:02.961540 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:02.961547 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:02.961633 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:02.962072 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:02.962079 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:02.962222 (+ 143us) client_negotiation.cc:769] Sending connection context
0811 05:16:02.962393 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 05:16:02.962587 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":147,"threads_started":1}
I20260811 05:16:02.962774 28961 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } attempt: 3
I20260811 05:16:02.964596 29193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:16:02 dist-test-slave-04bb krb5kdc[28832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425362, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for kudu/127.7.67.188@KRBTEST.COM
Aug 11 05:16:02 dist-test-slave-04bb krb5kdc[28832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
Aug 11 05:16:02 dist-test-slave-04bb krb5kdc[28832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425362, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.188@KRBTEST.COM for kudu/127.7.67.189@KRBTEST.COM
I20260811 05:16:02.973526 28965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.963839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:45433 (local address 127.0.0.1:41858)
0811 05:16:02.964016 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.964031 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:02.964110 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:02.964497 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:02.964501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:02.964710 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:02.964814 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:02.964823 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.965885 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.965891 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:02.966584 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:02.966594 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.967211 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.967215 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:02.967301 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:02.967821 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:02.967841 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:02.969594 (+ 1753us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:02.971561 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:02.971567 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:02.971571 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:02.971854 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:02.972390 (+ 536us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:02.972393 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:02.972395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:02.972487 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:02.973037 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:02.973040 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:02.973150 (+ 110us) client_negotiation.cc:769] Sending connection context
0811 05:16:02.973240 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 05:16:02.973389 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
I20260811 05:16:02.974124 29196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.964121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41858 (local address 127.7.67.188:45433)
0811 05:16:02.964235 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:02.964238 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:02.964286 (+ 48us) server_negotiation.cc:407] Connection header received
0811 05:16:02.964323 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:02.964327 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:02.964368 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:02.964456 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:02.964946 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.965752 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.966725 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.967063 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.967194 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:02.969782 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:02.969790 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:02.969793 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:02.969827 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:02.971438 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:02.972217 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:02.972220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:02.972221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:02.972292 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:02.972675 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:02.972680 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:02.972684 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:02.972908 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:02.973708 (+ 800us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:02.973801 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 05:16:02.973979 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 05:16:02.975268 29197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.965132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:41025 (local address 127.0.0.1:41648)
0811 05:16:02.965467 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.965479 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:02.965580 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:02.965826 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:02.965829 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:02.966006 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:02.966138 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:02.966146 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.967136 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.967140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:02.967998 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:02.968015 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.968702 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.968706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:02.968824 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:02.969362 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:02.969379 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:02.971659 (+ 2280us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:02.973687 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:02.973691 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:02.973693 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:02.974007 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:02.974338 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:02.974340 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:02.974342 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:02.974432 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:02.974810 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:02.974814 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:02.974906 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 05:16:02.975011 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 05:16:02.975136 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270}
I20260811 05:16:02.975688 29199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.962715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58822 (local address 127.7.67.190:39699)
0811 05:16:02.963085 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:02.963101 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:02.963412 (+ 311us) server_negotiation.cc:407] Connection header received
0811 05:16:02.963525 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:02.963528 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:02.963570 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:02.963725 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:02.965046 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.965846 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.966892 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.967187 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.967341 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:02.971355 (+ 4014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:02.971361 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:02.971365 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:02.971396 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:02.973282 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:02.973819 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:02.973823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:02.973825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:02.973919 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:02.974311 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:02.974317 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:02.974318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:02.974543 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:02.974760 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:02.975441 (+ 681us) server_negotiation.cc:299] Negotiation successful
0811 05:16:02.975555 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":79,"threads_started":1}
I20260811 05:16:02.976907 29081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.965249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41648 (local address 127.7.67.189:41025)
0811 05:16:02.965382 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:02.965387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:02.965544 (+ 157us) server_negotiation.cc:407] Connection header received
0811 05:16:02.965670 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:02.965673 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:02.965725 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:02.965825 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:02.966285 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.967013 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.968148 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.968408 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.968530 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:02.971798 (+ 3268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:02.971805 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:02.971808 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:02.971844 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:02.973536 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:02.974124 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:02.974128 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:02.974130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:02.974226 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:02.974520 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:02.974523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:02.974525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:02.974702 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:02.974820 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:02.976625 (+ 1805us) server_negotiation.cc:299] Negotiation successful
0811 05:16:02.976765 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:16:02.977124 29073 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } attempt: 3
I20260811 05:16:02.977785 29193 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:16:02.977957 28961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36b73c3b875046b48df6053302ccd4e7: Bootstrap starting.
I20260811 05:16:02.981186 28961 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36b73c3b875046b48df6053302ccd4e7: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:02.982285 28961 log.cc:824] T 00000000000000000000000000000000 P 36b73c3b875046b48df6053302ccd4e7: Log is configured to *not* fsync() on all Append() calls
Aug 11 05:16:02 dist-test-slave-04bb krb5kdc[28832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425362, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.189@KRBTEST.COM for kudu/127.7.67.188@KRBTEST.COM
Aug 11 05:16:02 dist-test-slave-04bb krb5kdc[28832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425362, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.188@KRBTEST.COM for kudu/127.7.67.188@KRBTEST.COM
I20260811 05:16:02.984843 28961 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36b73c3b875046b48df6053302ccd4e7: No bootstrap required, opened a new log
Aug 11 05:16:02 dist-test-slave-04bb krb5kdc[28832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425362, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.189@KRBTEST.COM
I20260811 05:16:02.988370 29078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.977517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:45433 (local address 127.0.0.1:41862)
0811 05:16:02.977674 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.977689 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:02.977766 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:02.978045 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:02.978048 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:02.978214 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:02.978312 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:02.978319 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.979523 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.979527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:02.980279 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:02.980291 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.980900 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.980904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:02.980998 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:02.981546 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:02.981564 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:02.983691 (+ 2127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:02.985952 (+ 2261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:02.985956 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:02.985958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:02.986215 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:02.986576 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:02.986579 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:02.986582 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:02.986687 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:02.987427 (+ 740us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:02.987431 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:02.987540 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 05:16:02.987947 (+ 407us) client_negotiation.cc:240] Negotiation successful
0811 05:16:02.988142 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 05:16:02.988492 29196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.977580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41862 (local address 127.7.67.188:45433)
0811 05:16:02.977702 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:02.977706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:02.977723 (+ 17us) server_negotiation.cc:407] Connection header received
0811 05:16:02.977861 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:02.977864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:02.977906 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:02.978429 (+ 523us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:02.978510 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.979414 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.980429 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.980764 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.981272 (+ 508us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:02.983846 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:02.983850 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:02.983853 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:02.983871 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:02.985830 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:02.986368 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:02.986372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:02.986374 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:02.986463 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:02.986802 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:02.986806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:02.986811 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:02.987052 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:02.987163 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:02.988190 (+ 1027us) server_negotiation.cc:299] Negotiation successful
0811 05:16:02.988304 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":62}
I20260811 05:16:02.989171 28961 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36b73c3b875046b48df6053302ccd4e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b73c3b875046b48df6053302ccd4e7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39699 } } peers { permanent_uuid: "8af95784b7e84f459ca1daf81e7bcc26" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } } peers { permanent_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } }
I20260811 05:16:02.989460 28961 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36b73c3b875046b48df6053302ccd4e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:02.989567 28961 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36b73c3b875046b48df6053302ccd4e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36b73c3b875046b48df6053302ccd4e7, State: Initialized, Role: FOLLOWER
I20260811 05:16:02.989815 29197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.978229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:45433 (local address 127.0.0.1:41870)
0811 05:16:02.978325 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:02.978340 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:02.978408 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:02.978907 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:02.978912 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:02.979092 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:02.979268 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:02.979283 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.980299 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.980302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:02.981061 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:02.981072 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.981704 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.981708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:02.981778 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:02.982242 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:02.982259 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:02.983662 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:02.988235 (+ 4573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:02.988237 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:02.988240 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:02.988488 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:02.988840 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:02.988846 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:02.988851 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:02.988935 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:02.989338 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:02.989340 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:02.989431 (+ 91us) client_negotiation.cc:769] Sending connection context
0811 05:16:02.989532 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 05:16:02.989672 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":18}
I20260811 05:16:02.990167 28961 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36b73c3b875046b48df6053302ccd4e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b73c3b875046b48df6053302ccd4e7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39699 } } peers { permanent_uuid: "8af95784b7e84f459ca1daf81e7bcc26" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } } peers { permanent_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } }
I20260811 05:16:02.990401 29209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.978398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41870 (local address 127.7.67.188:45433)
0811 05:16:02.978681 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:02.978690 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:02.978702 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:16:02.978737 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:02.978740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:02.978782 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:02.979232 (+ 450us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:02.979385 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.980191 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.981198 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.981593 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.981983 (+ 390us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:02.983817 (+ 1834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:02.983822 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:02.983826 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:02.983850 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:02.988122 (+ 4272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:02.988643 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:02.988647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:02.988651 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:02.988745 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:02.989015 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:02.989018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:02.989020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:02.989204 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:02.989305 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:02.990112 (+ 807us) server_negotiation.cc:299] Negotiation successful
0811 05:16:02.990233 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":219,"thread_start_us":121,"threads_started":1}
I20260811 05:16:02.990715 29211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36b73c3b875046b48df6053302ccd4e7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b73c3b875046b48df6053302ccd4e7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39699 } } peers { permanent_uuid: "8af95784b7e84f459ca1daf81e7bcc26" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } } peers { permanent_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } } }
I20260811 05:16:02.990875 29211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36b73c3b875046b48df6053302ccd4e7 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:16:02.991616 28961 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36b73c3b875046b48df6053302ccd4e7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:16:02.992343 29073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8af95784b7e84f459ca1daf81e7bcc26: Bootstrap starting.
I20260811 05:16:02.993324 29193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a: Bootstrap starting.
I20260811 05:16:02.995193 29073 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8af95784b7e84f459ca1daf81e7bcc26: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:02.995851 29193 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:02.996649 29193 log.cc:824] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:02.996623 29073 log.cc:824] T 00000000000000000000000000000000 P 8af95784b7e84f459ca1daf81e7bcc26: Log is configured to *not* fsync() on all Append() calls
W20260811 05:16:02.997547 29231 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 36b73c3b875046b48df6053302ccd4e7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:16:02.997629 29231 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:16:02.999608 29073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8af95784b7e84f459ca1daf81e7bcc26: No bootstrap required, opened a new log
I20260811 05:16:02.999825 29193 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a: No bootstrap required, opened a new log
I20260811 05:16:02.996714 28851 master_runner.cc:186] Error getting master registration for 127.7.67.189:41025: OK, instance_id { permanent_uuid: "8af95784b7e84f459ca1daf81e7bcc26" instance_seqno: 1786425362721180 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 05:16:03.000442 28851 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 05:16:03.000705 29200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:02.977071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41654 (local address 127.7.67.189:41025)
0811 05:16:02.978728 (+ 1657us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:02.978737 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:02.978748 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:16:02.978777 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:02.978780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:02.978824 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:02.979660 (+ 836us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:02.979803 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.980573 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.982282 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:02.982707 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:02.982849 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:02.987255 (+ 4406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:02.987258 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:02.987260 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:02.987303 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:02.997968 (+ 10665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:02.998567 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:02.998571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:02.998573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:02.998670 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:02.999013 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:02.999016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:02.999018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:02.999244 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:03.000053 (+ 809us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:03.000139 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 05:16:03.000419 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1533,"thread_start_us":108,"threads_started":1}
I20260811 05:16:03.004060 29193 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b73c3b875046b48df6053302ccd4e7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39699 } } peers { permanent_uuid: "8af95784b7e84f459ca1daf81e7bcc26" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } } peers { permanent_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } }
I20260811 05:16:03.004376 29193 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:03.004448 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:16:03.004472 29193 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d1cebc1b8f0424b926d80ab3cad2f3a, State: Initialized, Role: FOLLOWER
I20260811 05:16:03.004881 29073 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8af95784b7e84f459ca1daf81e7bcc26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b73c3b875046b48df6053302ccd4e7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39699 } } peers { permanent_uuid: "8af95784b7e84f459ca1daf81e7bcc26" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } } peers { permanent_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } }
I20260811 05:16:03.005146 29073 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8af95784b7e84f459ca1daf81e7bcc26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:03.005072 29193 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b73c3b875046b48df6053302ccd4e7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39699 } } peers { permanent_uuid: "8af95784b7e84f459ca1daf81e7bcc26" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } } peers { permanent_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } }
I20260811 05:16:03.005200 29073 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8af95784b7e84f459ca1daf81e7bcc26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8af95784b7e84f459ca1daf81e7bcc26, State: Initialized, Role: FOLLOWER
I20260811 05:16:03.005694 29073 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8af95784b7e84f459ca1daf81e7bcc26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b73c3b875046b48df6053302ccd4e7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39699 } } peers { permanent_uuid: "8af95784b7e84f459ca1daf81e7bcc26" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } } peers { permanent_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } }
I20260811 05:16:03.006583 29193 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:16:03.007146 29073 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8af95784b7e84f459ca1daf81e7bcc26 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:16:03.010470 29243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8af95784b7e84f459ca1daf81e7bcc26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b73c3b875046b48df6053302ccd4e7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39699 } } peers { permanent_uuid: "8af95784b7e84f459ca1daf81e7bcc26" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } } peers { permanent_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } } }
I20260811 05:16:03.010689 29243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8af95784b7e84f459ca1daf81e7bcc26 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 05:16:03.012692 29274 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:16:03.012848 29274 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 05:16:03.013382 29288 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8af95784b7e84f459ca1daf81e7bcc26: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:16:03.013537 29288 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:16:03.014161 29240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b73c3b875046b48df6053302ccd4e7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39699 } } peers { permanent_uuid: "8af95784b7e84f459ca1daf81e7bcc26" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } } peers { permanent_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } } }
I20260811 05:16:03.014381 29240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:16:03.018375 29084 master_runner.cc:263] No leader master found from master 8d1cebc1b8f0424b926d80ab3cad2f3a
I20260811 05:16:03.018451 29084 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 05:16:03.020020 28970 master_runner.cc:263] No leader master found from master 8af95784b7e84f459ca1daf81e7bcc26
I20260811 05:16:03.020088 28970 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 05:16:03.028352 29209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:03.012213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41876 (local address 127.7.67.188:45433)
0811 05:16:03.012330 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:03.012335 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:03.013955 (+ 1620us) server_negotiation.cc:407] Connection header received
0811 05:16:03.014007 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:03.014010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:03.014063 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:03.014169 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:03.015164 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:03.016007 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:03.017338 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:03.017712 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:03.017794 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:03.023930 (+ 6136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:03.023934 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:03.023937 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:03.023966 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:03.026068 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:03.026913 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:03.026917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:03.026919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:03.027014 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:03.027321 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:03.027324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:03.027326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:03.027547 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:03.027661 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:03.028038 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 05:16:03.028179 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 05:16:03.032222 29199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:03.011229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58832 (local address 127.7.67.190:39699)
0811 05:16:03.011363 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:03.011369 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:03.013518 (+ 2149us) server_negotiation.cc:407] Connection header received
0811 05:16:03.013693 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:03.013697 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:03.013755 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:03.013839 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:03.020498 (+ 6659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:03.021337 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:03.022603 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:03.022976 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:03.023092 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:03.025813 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:03.025818 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:03.025821 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:03.025852 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:03.028023 (+ 2171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:03.029343 (+ 1320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:03.029347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:03.029349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:03.029456 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:03.029813 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:03.029816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:03.029818 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:03.030057 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:03.030251 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:03.031257 (+ 1006us) server_negotiation.cc:299] Negotiation successful
0811 05:16:03.031724 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 05:16:03.034809 29200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:03.013019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41662 (local address 127.7.67.189:41025)
0811 05:16:03.013164 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:03.013171 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:03.018768 (+ 5597us) server_negotiation.cc:407] Connection header received
0811 05:16:03.019955 (+ 1187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:03.019959 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:03.020010 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:03.020067 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:03.021912 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:03.022672 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:03.024372 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:03.025149 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:03.025231 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:03.030945 (+ 5714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:03.030949 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:03.030953 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:03.030984 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:03.032859 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:03.033402 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:03.033406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:03.033408 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:03.033498 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:03.033811 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:03.033814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:03.033816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:03.034031 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:03.034153 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:03.034504 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 05:16:03.034636 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 05:16:03.050678 29240 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:16:03.050911 29240 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b73c3b875046b48df6053302ccd4e7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39699 } } peers { permanent_uuid: "8af95784b7e84f459ca1daf81e7bcc26" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } } peers { permanent_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } }
I20260811 05:16:03.052070 29240 leader_election.cc:288] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36b73c3b875046b48df6053302ccd4e7 (127.7.67.190:39699), 8af95784b7e84f459ca1daf81e7bcc26 (127.7.67.189:41025)
I20260811 05:16:03.052556 28922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36b73c3b875046b48df6053302ccd4e7" is_pre_election: true
I20260811 05:16:03.052835 29031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8af95784b7e84f459ca1daf81e7bcc26" is_pre_election: true
I20260811 05:16:03.052915 28922 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 36b73c3b875046b48df6053302ccd4e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d1cebc1b8f0424b926d80ab3cad2f3a in term 0.
I20260811 05:16:03.053149 29031 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8af95784b7e84f459ca1daf81e7bcc26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8d1cebc1b8f0424b926d80ab3cad2f3a in term 0.
I20260811 05:16:03.053640 29120 leader_election.cc:302] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8af95784b7e84f459ca1daf81e7bcc26, 8d1cebc1b8f0424b926d80ab3cad2f3a; no voters:
I20260811 05:16:03.054253 29240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:16:03.054347 29240 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:16:03.054396 29240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:03.055783 29240 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b73c3b875046b48df6053302ccd4e7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39699 } } peers { permanent_uuid: "8af95784b7e84f459ca1daf81e7bcc26" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } } peers { permanent_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } }
I20260811 05:16:03.056638 28922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36b73c3b875046b48df6053302ccd4e7"
I20260811 05:16:03.056772 28922 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36b73c3b875046b48df6053302ccd4e7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:03.058166 28922 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 36b73c3b875046b48df6053302ccd4e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1cebc1b8f0424b926d80ab3cad2f3a in term 1.
I20260811 05:16:03.058595 29116 leader_election.cc:302] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36b73c3b875046b48df6053302ccd4e7, 8d1cebc1b8f0424b926d80ab3cad2f3a; no voters:
I20260811 05:16:03.059105 29240 leader_election.cc:288] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [CANDIDATE]: Term 1 election: Requested vote from peers 36b73c3b875046b48df6053302ccd4e7 (127.7.67.190:39699), 8af95784b7e84f459ca1daf81e7bcc26 (127.7.67.189:41025)
I20260811 05:16:03.059582 29031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8af95784b7e84f459ca1daf81e7bcc26"
I20260811 05:16:03.059721 29031 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8af95784b7e84f459ca1daf81e7bcc26 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:03.061196 29031 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8af95784b7e84f459ca1daf81e7bcc26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8d1cebc1b8f0424b926d80ab3cad2f3a in term 1.
I20260811 05:16:03.061712 29240 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:16:03.061859 29240 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [term 1 LEADER]: Becoming Leader. State: Replica: 8d1cebc1b8f0424b926d80ab3cad2f3a, State: Running, Role: LEADER
I20260811 05:16:03.062275 29240 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b73c3b875046b48df6053302ccd4e7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39699 } } peers { permanent_uuid: "8af95784b7e84f459ca1daf81e7bcc26" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } } peers { permanent_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } }
I20260811 05:16:03.064400 29240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d1cebc1b8f0424b926d80ab3cad2f3a. Latest consensus state: current_term: 1 leader_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b73c3b875046b48df6053302ccd4e7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39699 } } peers { permanent_uuid: "8af95784b7e84f459ca1daf81e7bcc26" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } } peers { permanent_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } } }
I20260811 05:16:03.064612 29240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [sys.catalog]: This master's current role is: LEADER
I20260811 05:16:03.065847 29298 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:16:03.066931 29298 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:16:03.074416 29031 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8af95784b7e84f459ca1daf81e7bcc26 [term 1 FOLLOWER]: Refusing update from remote peer 8d1cebc1b8f0424b926d80ab3cad2f3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:16:03.075345 29296 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8af95784b7e84f459ca1daf81e7bcc26" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:16:03.080471 29243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8af95784b7e84f459ca1daf81e7bcc26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d1cebc1b8f0424b926d80ab3cad2f3a. Latest consensus state: current_term: 1 leader_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b73c3b875046b48df6053302ccd4e7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39699 } } peers { permanent_uuid: "8af95784b7e84f459ca1daf81e7bcc26" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } } peers { permanent_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } } }
I20260811 05:16:03.080662 29243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8af95784b7e84f459ca1daf81e7bcc26 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:16:03.081874 28922 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 36b73c3b875046b48df6053302ccd4e7 [term 1 FOLLOWER]: Refusing update from remote peer 8d1cebc1b8f0424b926d80ab3cad2f3a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:16:03.082561 29296 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [LEADER]: Connected to new peer: Peer: permanent_uuid: "36b73c3b875046b48df6053302ccd4e7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39699 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:16:03.086913 29211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36b73c3b875046b48df6053302ccd4e7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d1cebc1b8f0424b926d80ab3cad2f3a. Latest consensus state: current_term: 1 leader_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b73c3b875046b48df6053302ccd4e7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39699 } } peers { permanent_uuid: "8af95784b7e84f459ca1daf81e7bcc26" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } } peers { permanent_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } } }
I20260811 05:16:03.087040 29211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36b73c3b875046b48df6053302ccd4e7 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:16:03.094544 29296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b73c3b875046b48df6053302ccd4e7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39699 } } peers { permanent_uuid: "8af95784b7e84f459ca1daf81e7bcc26" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } } peers { permanent_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } } }
I20260811 05:16:03.094681 29296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [sys.catalog]: This master's current role is: LEADER
I20260811 05:16:03.096145 29211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36b73c3b875046b48df6053302ccd4e7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b73c3b875046b48df6053302ccd4e7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39699 } } peers { permanent_uuid: "8af95784b7e84f459ca1daf81e7bcc26" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } } peers { permanent_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } } }
I20260811 05:16:03.096263 29211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36b73c3b875046b48df6053302ccd4e7 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:16:03.096814 29300 mvcc.cc:205] Tried to move back new op lower bound from 7317198287144480768 to 7317198287107059712. Current Snapshot: MvccSnapshot[applied={T|T < 7317198287144480768}]
I20260811 05:16:03.098744 29304 mvcc.cc:205] Tried to move back new op lower bound from 7317198287144480768 to 7317198287107059712. Current Snapshot: MvccSnapshot[applied={T|T < 7317198287144480768}]
I20260811 05:16:03.099514 29240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b73c3b875046b48df6053302ccd4e7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39699 } } peers { permanent_uuid: "8af95784b7e84f459ca1daf81e7bcc26" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } } peers { permanent_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } } }
I20260811 05:16:03.099625 29240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a [sys.catalog]: This master's current role is: LEADER
I20260811 05:16:03.102257 29243 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8af95784b7e84f459ca1daf81e7bcc26 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "36b73c3b875046b48df6053302ccd4e7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 39699 } } peers { permanent_uuid: "8af95784b7e84f459ca1daf81e7bcc26" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 41025 } } peers { permanent_uuid: "8d1cebc1b8f0424b926d80ab3cad2f3a" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 45433 } } }
I20260811 05:16:03.102387 29243 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8af95784b7e84f459ca1daf81e7bcc26 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:16:03.102622 29298 catalog_manager.cc:1347] Generated new cluster ID: e1b8b4fb9b924ce5af18336a2c8ab570
I20260811 05:16:03.102672 29298 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:16:03.104449 29301 mvcc.cc:205] Tried to move back new op lower bound from 7317198287144480768 to 7317198287107059712. Current Snapshot: MvccSnapshot[applied={T|T < 7317198287144480768}]
I20260811 05:16:03.125479 29298 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:16:03.126811 29298 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:16:03.141054 29298 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d1cebc1b8f0424b926d80ab3cad2f3a: Generated new TSK 0
I20260811 05:16:03.141923 29298 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:16:03.999018 29231 catalog_manager.cc:1259] Loaded cluster ID: e1b8b4fb9b924ce5af18336a2c8ab570
I20260811 05:16:03.999106 29231 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 36b73c3b875046b48df6053302ccd4e7: loading cluster ID for follower catalog manager: success
I20260811 05:16:04.000941 29231 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 36b73c3b875046b48df6053302ccd4e7: acquiring CA information for follower catalog manager: success
I20260811 05:16:04.002115 29231 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 36b73c3b875046b48df6053302ccd4e7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 05:16:04.014988 29288 catalog_manager.cc:1259] Loaded cluster ID: e1b8b4fb9b924ce5af18336a2c8ab570
I20260811 05:16:04.015075 29288 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8af95784b7e84f459ca1daf81e7bcc26: loading cluster ID for follower catalog manager: success
I20260811 05:16:04.016808 29288 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8af95784b7e84f459ca1daf81e7bcc26: acquiring CA information for follower catalog manager: success
I20260811 05:16:04.017966 29288 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8af95784b7e84f459ca1daf81e7bcc26: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 05:16:04.208077 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 28851
I20260811 05:16:04.224959 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 28970
I20260811 05:16:04.239487 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 29084
2026-08-11T05:16:04Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3365 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:16:04 dist-test-slave-04bb krb5kdc[29321](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:16:04 dist-test-slave-04bb krb5kdc[29321](info): set up 2 sockets
Aug 11 05:16:04 dist-test-slave-04bb krb5kdc[29321](info): commencing operation
krb5kdc: starting...
W20260811 05:16:05.562862 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.291s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:16:05 dist-test-slave-04bb krb5kdc[29321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:16:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:16:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:16:05.698056 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:34347
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:39151
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:34347
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:16:05.817879 29346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:05.818193 29346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:05.818271 29346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:05.822324 29346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:16:05.822417 29346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:05.822451 29346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:05.822491 29346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:16:05.822526 29346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:16:05.827822 29346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:39151
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:34347
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:34347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:16:05.828994 29346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:05.830065 29346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:16:05.838933 29346 server_base.cc:1034] running on GCE node
W20260811 05:16:05.838901 29357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:05.838892 29354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:05.838963 29352 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:05.839809 29346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:05.841013 29346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:05.842211 29346 hybrid_clock.cc:648] HybridClock initialized: now 1786425365842184 us; error 41 us; skew 500 ppm
Aug 11 05:16:05 dist-test-slave-04bb krb5kdc[29321](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425365, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:05.845191 29346 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:16:05.846153 29346 webserver.cc:460] Webserver started at http://127.7.67.190:42545/ using document root <none> and password file <none>
I20260811 05:16:05.846743 29346 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:05.846834 29346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:05.847074 29346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:16:05.849442 29346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "3b75309618ee461dae984c6231017cac"
format_stamp: "Formatted at 2026-08-11 05:16:05 on dist-test-slave-04bb"
server_key: "aeab0da3a2a0abfd059eb37a491720b9"
server_key_iv: "b55657023a19e236e387ee8bea5a0c0e"
server_key_version: "encryptionkey@0"
I20260811 05:16:05.850045 29346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "3b75309618ee461dae984c6231017cac"
format_stamp: "Formatted at 2026-08-11 05:16:05 on dist-test-slave-04bb"
server_key: "aeab0da3a2a0abfd059eb37a491720b9"
server_key_iv: "b55657023a19e236e387ee8bea5a0c0e"
server_key_version: "encryptionkey@0"
I20260811 05:16:05.854173 29346 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 05:16:05.856834 29372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:05.858146 29346 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:16:05.858320 29346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "3b75309618ee461dae984c6231017cac"
format_stamp: "Formatted at 2026-08-11 05:16:05 on dist-test-slave-04bb"
server_key: "aeab0da3a2a0abfd059eb37a491720b9"
server_key_iv: "b55657023a19e236e387ee8bea5a0c0e"
server_key_version: "encryptionkey@0"
I20260811 05:16:05.858462 29346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:05.892354 29346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:05.896014 29346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:05.896250 29346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:05.904042 29346 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:34347
I20260811 05:16:05.904072 29447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:34347 every 8 connection(s)
I20260811 05:16:05.905113 29346 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:16:05.906276 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 29346
I20260811 05:16:05.906410 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:16:05.906737 7438 external_mini_cluster.cc:1428] Setting key 84812789888a81d72fb49950633d0a93
I20260811 05:16:05.908694 29448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:05.914477 29448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b75309618ee461dae984c6231017cac: Bootstrap starting.
Aug 11 05:16:05 dist-test-slave-04bb krb5kdc[29321](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425365, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:16:05.917074 29448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3b75309618ee461dae984c6231017cac: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:05.917874 29448 log.cc:824] T 00000000000000000000000000000000 P 3b75309618ee461dae984c6231017cac: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:05.919996 29448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3b75309618ee461dae984c6231017cac: No bootstrap required, opened a new log
I20260811 05:16:05.922093 29451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:05.908336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46584 (local address 127.7.67.190:34347)
0811 05:16:05.908877 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:05.908895 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:05.908936 (+ 41us) server_negotiation.cc:407] Connection header received
0811 05:16:05.909681 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:05.909700 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:05.910058 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:05.910455 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:05.911390 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:05.912501 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:05.913492 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:05.914045 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:05.914254 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:05.917229 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:05.917237 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:05.917256 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:05.917296 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:05.919222 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:05.919697 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:05.919703 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:05.919708 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:05.919826 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:05.920118 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:05.920125 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:05.920127 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:05.920575 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:05.920793 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:05.921054 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 05:16:05.921358 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":168,"threads_started":1}
I20260811 05:16:05.924865 29448 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3b75309618ee461dae984c6231017cac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b75309618ee461dae984c6231017cac" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 34347 } }
I20260811 05:16:05.925221 29448 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3b75309618ee461dae984c6231017cac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:05.925307 29448 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3b75309618ee461dae984c6231017cac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b75309618ee461dae984c6231017cac, State: Initialized, Role: FOLLOWER
I20260811 05:16:05.925896 29448 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3b75309618ee461dae984c6231017cac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b75309618ee461dae984c6231017cac" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 34347 } }
I20260811 05:16:05.925989 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:16:05.926093 29448 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3b75309618ee461dae984c6231017cac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:16:05.926198 29448 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3b75309618ee461dae984c6231017cac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:16:05.926347 29448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3b75309618ee461dae984c6231017cac [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:05.927582 29448 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3b75309618ee461dae984c6231017cac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b75309618ee461dae984c6231017cac" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 34347 } }
I20260811 05:16:05.927995 29448 leader_election.cc:302] T 00000000000000000000000000000000 P 3b75309618ee461dae984c6231017cac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3b75309618ee461dae984c6231017cac; no voters:
I20260811 05:16:05.928370 29448 leader_election.cc:288] T 00000000000000000000000000000000 P 3b75309618ee461dae984c6231017cac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:16:05.928476 29453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3b75309618ee461dae984c6231017cac [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:16:05.928769 29453 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3b75309618ee461dae984c6231017cac [term 1 LEADER]: Becoming Leader. State: Replica: 3b75309618ee461dae984c6231017cac, State: Running, Role: LEADER
I20260811 05:16:05.929138 29453 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3b75309618ee461dae984c6231017cac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b75309618ee461dae984c6231017cac" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 34347 } }
I20260811 05:16:05.929813 29448 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3b75309618ee461dae984c6231017cac [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:16:05.930907 29456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b75309618ee461dae984c6231017cac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3b75309618ee461dae984c6231017cac. Latest consensus state: current_term: 1 leader_uuid: "3b75309618ee461dae984c6231017cac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b75309618ee461dae984c6231017cac" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 34347 } } }
I20260811 05:16:05.931056 29456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b75309618ee461dae984c6231017cac [sys.catalog]: This master's current role is: LEADER
I20260811 05:16:05.931360 29454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3b75309618ee461dae984c6231017cac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3b75309618ee461dae984c6231017cac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3b75309618ee461dae984c6231017cac" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 34347 } } }
I20260811 05:16:05.931440 29475 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:16:05.931509 29454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3b75309618ee461dae984c6231017cac [sys.catalog]: This master's current role is: LEADER
I20260811 05:16:05.934764 29475 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:16:05.940570 29475 catalog_manager.cc:1347] Generated new cluster ID: 5eae3ad6bb1943df8cfe6a92c3bbba41
I20260811 05:16:05.940649 29475 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:16:05.950336 29475 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:16:05.951606 29475 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:16:05.964558 29475 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3b75309618ee461dae984c6231017cac: Generated new TSK 0
I20260811 05:16:05.965461 29475 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:16:05.982235 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 29346
2026-08-11T05:16:06Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1756 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:16:06 dist-test-slave-04bb krb5kdc[29505](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:16:06 dist-test-slave-04bb krb5kdc[29505](info): set up 2 sockets
Aug 11 05:16:06 dist-test-slave-04bb krb5kdc[29505](info): commencing operation
krb5kdc: starting...
W20260811 05:16:07.329938 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:16:07 dist-test-slave-04bb krb5kdc[29505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:16:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:16:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:16:07.463740 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:38739
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:38195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:38739
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:16:07.581022 29527 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:07.581290 29527 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:07.581357 29527 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:07.585367 29527 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:16:07.585427 29527 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:07.585446 29527 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:07.585464 29527 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:16:07.585479 29527 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:16:07.590801 29527 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:38195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:38739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:38739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29527
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:16:07.591825 29527 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:07.592904 29527 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:07.601677 29533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:07.601641 29540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:07.601663 29537 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:07.602017 29527 server_base.cc:1034] running on GCE node
I20260811 05:16:07.602620 29527 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:07.603744 29527 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:07.604904 29527 hybrid_clock.cc:648] HybridClock initialized: now 1786425367604889 us; error 29 us; skew 500 ppm
Aug 11 05:16:07 dist-test-slave-04bb krb5kdc[29505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425367, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:07.607703 29527 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:16:07.608690 29527 webserver.cc:460] Webserver started at http://127.7.67.190:33627/ using document root <none> and password file <none>
I20260811 05:16:07.609237 29527 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:07.609298 29527 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:07.609468 29527 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:16:07.611562 29527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "0c8a1f31a2f9487ab869de998b28f97e"
format_stamp: "Formatted at 2026-08-11 05:16:07 on dist-test-slave-04bb"
server_key: "b85f448a35f1f34828dc85dea019e4fb"
server_key_iv: "cda249c7ad42af3d947ec61529234837"
server_key_version: "encryptionkey@0"
I20260811 05:16:07.612083 29527 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "0c8a1f31a2f9487ab869de998b28f97e"
format_stamp: "Formatted at 2026-08-11 05:16:07 on dist-test-slave-04bb"
server_key: "b85f448a35f1f34828dc85dea019e4fb"
server_key_iv: "cda249c7ad42af3d947ec61529234837"
server_key_version: "encryptionkey@0"
I20260811 05:16:07.615886 29527 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 05:16:07.618409 29554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:07.619712 29527 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 05:16:07.619865 29527 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "0c8a1f31a2f9487ab869de998b28f97e"
format_stamp: "Formatted at 2026-08-11 05:16:07 on dist-test-slave-04bb"
server_key: "b85f448a35f1f34828dc85dea019e4fb"
server_key_iv: "cda249c7ad42af3d947ec61529234837"
server_key_version: "encryptionkey@0"
I20260811 05:16:07.619992 29527 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:07.646524 29527 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:07.649847 29527 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:07.650094 29527 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:07.658246 29527 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:38739
I20260811 05:16:07.658550 29633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:38739 every 8 connection(s)
I20260811 05:16:07.659428 29527 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:16:07.661515 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 29527
I20260811 05:16:07.661644 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:16:07.661991 7438 external_mini_cluster.cc:1428] Setting key 92756ea01fdbd96202f6aff48a33ced1
I20260811 05:16:07.663072 29637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:07.670125 29637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e: Bootstrap starting.
Aug 11 05:16:07 dist-test-slave-04bb krb5kdc[29505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425367, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:16:07.672969 29637 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:07.673910 29637 log.cc:824] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:07.675887 29637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e: No bootstrap required, opened a new log
I20260811 05:16:07.677563 29640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:07.663565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33122 (local address 127.7.67.190:38739)
0811 05:16:07.664025 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:07.664043 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:07.664084 (+ 41us) server_negotiation.cc:407] Connection header received
0811 05:16:07.664893 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:07.664913 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:07.665283 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:07.665585 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:07.666527 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:07.667629 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:07.668902 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:07.669419 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:07.669548 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:07.672593 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:07.672600 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:07.672616 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:07.672649 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:07.674645 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:07.675123 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:07.675129 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:07.675134 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:07.675258 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:07.675556 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:07.675560 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:07.675561 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:07.675968 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:07.676204 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:07.676506 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 05:16:07.676838 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":160,"threads_started":1}
I20260811 05:16:07.680012 29637 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8a1f31a2f9487ab869de998b28f97e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38739 } }
I20260811 05:16:07.680267 29637 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:07.680342 29637 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c8a1f31a2f9487ab869de998b28f97e, State: Initialized, Role: FOLLOWER
I20260811 05:16:07.680817 29637 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8a1f31a2f9487ab869de998b28f97e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38739 } }
I20260811 05:16:07.680975 29637 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:16:07.681057 29637 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:16:07.681157 29637 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:07.682060 29637 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8a1f31a2f9487ab869de998b28f97e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38739 } }
I20260811 05:16:07.682399 29637 leader_election.cc:302] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c8a1f31a2f9487ab869de998b28f97e; no voters:
I20260811 05:16:07.682729 29637 leader_election.cc:288] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:16:07.682879 29643 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:16:07.683135 29643 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 1 LEADER]: Becoming Leader. State: Replica: 0c8a1f31a2f9487ab869de998b28f97e, State: Running, Role: LEADER
I20260811 05:16:07.683517 29643 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8a1f31a2f9487ab869de998b28f97e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38739 } }
I20260811 05:16:07.684028 29637 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:16:07.685330 29644 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c8a1f31a2f9487ab869de998b28f97e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8a1f31a2f9487ab869de998b28f97e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38739 } } }
I20260811 05:16:07.685392 29646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c8a1f31a2f9487ab869de998b28f97e. Latest consensus state: current_term: 1 leader_uuid: "0c8a1f31a2f9487ab869de998b28f97e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8a1f31a2f9487ab869de998b28f97e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38739 } } }
I20260811 05:16:07.685468 29644 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [sys.catalog]: This master's current role is: LEADER
I20260811 05:16:07.685503 29646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [sys.catalog]: This master's current role is: LEADER
I20260811 05:16:07.685889 29659 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:16:07.689241 29659 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:16:07.698021 29659 catalog_manager.cc:1347] Generated new cluster ID: 49ebe35220b74d2f905142fc77f917f8
I20260811 05:16:07.698146 29659 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:16:07.710398 29659 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:16:07.711645 29659 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:16:07.718340 29659 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e: Generated new TSK 0
I20260811 05:16:07.719049 29659 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
I20260811 05:16:07.760834 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:38739
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:38195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:16:07.901571 29681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:07.901851 29681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:07.901896 29681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:07.905947 29681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:07.906015 29681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:07.906109 29681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:16:07.911583 29681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:38195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:38739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:16:07.912806 29681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:07.913894 29681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:07.924198 29694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:07.924207 29695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:07.924417 29681 server_base.cc:1034] running on GCE node
W20260811 05:16:07.924196 29698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:07.924950 29681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:07.925638 29681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:07.926805 29681 hybrid_clock.cc:648] HybridClock initialized: now 1786425367926789 us; error 30 us; skew 500 ppm
Aug 11 05:16:07 dist-test-slave-04bb krb5kdc[29505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425367, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:07.929840 29681 init.cc:377] Logged in from keytab as kudu/127.7.67.129@KRBTEST.COM (short username kudu)
I20260811 05:16:07.930768 29681 webserver.cc:460] Webserver started at http://127.7.67.129:42163/ using document root <none> and password file <none>
I20260811 05:16:07.931355 29681 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:07.931452 29681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:07.931762 29681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:16:07.934546 29681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "bdabab5544a445619a9e3ceee6af4e6b"
format_stamp: "Formatted at 2026-08-11 05:16:07 on dist-test-slave-04bb"
server_key: "342143cf6cd5e69411c1f1b3e3297ea0"
server_key_iv: "f9dcdc70efd59352e1170b5035a9a2fc"
server_key_version: "encryptionkey@0"
I20260811 05:16:07.935256 29681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "bdabab5544a445619a9e3ceee6af4e6b"
format_stamp: "Formatted at 2026-08-11 05:16:07 on dist-test-slave-04bb"
server_key: "342143cf6cd5e69411c1f1b3e3297ea0"
server_key_iv: "f9dcdc70efd59352e1170b5035a9a2fc"
server_key_version: "encryptionkey@0"
I20260811 05:16:07.939271 29681 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 05:16:07.941919 29706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:07.943199 29681 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:16:07.943327 29681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "bdabab5544a445619a9e3ceee6af4e6b"
format_stamp: "Formatted at 2026-08-11 05:16:07 on dist-test-slave-04bb"
server_key: "342143cf6cd5e69411c1f1b3e3297ea0"
server_key_iv: "f9dcdc70efd59352e1170b5035a9a2fc"
server_key_version: "encryptionkey@0"
I20260811 05:16:07.943437 29681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:07.952953 29681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:07.956060 29681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:07.956259 29681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:07.956979 29681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:07.957835 29681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:16:07.957882 29681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:07.957947 29681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:16:07.957979 29681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:07.967300 29681 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:42341
I20260811 05:16:07.967332 29886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:42341 every 8 connection(s)
I20260811 05:16:07.968245 29681 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:16:07.969496 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 29681
I20260811 05:16:07.969632 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:16:07.969955 7438 external_mini_cluster.cc:1428] Setting key 1e0b69e546ffccbe3bebdb99c903548a
Aug 11 05:16:07 dist-test-slave-04bb krb5kdc[29505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425367, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.130@KRBTEST.COM" created.
I20260811 05:16:07.986008 29640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:07.970372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:56501 (local address 127.7.67.190:38739)
0811 05:16:07.970549 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:07.970554 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:07.971555 (+ 1001us) server_negotiation.cc:407] Connection header received
0811 05:16:07.973667 (+ 2112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:07.973670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:07.973718 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:07.973784 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:07.975215 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:07.975987 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:07.978050 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:07.978320 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:07.978416 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:07.981581 (+ 3165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:07.981590 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:07.981592 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:07.981618 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:07.983319 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:07.984000 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:07.984003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:07.984005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:07.984083 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:07.984432 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:07.984435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:07.984437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:07.984726 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:07.984829 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:07.985580 (+ 751us) server_negotiation.cc:299] Negotiation successful
0811 05:16:07.985784 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 05:16:07.986819 29894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:07.970817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.129:56501)
0811 05:16:07.971392 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:07.971457 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:07.973413 (+ 1956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:07.973970 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:07.973983 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:07.974509 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:07.975049 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:07.975064 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:07.976200 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:07.976207 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:07.977904 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:07.977922 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:07.978516 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:07.978523 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:07.978790 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:07.979521 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:07.979551 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:07.981409 (+ 1858us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:07.983496 (+ 2087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:07.983505 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:07.983525 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:07.983873 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:07.984177 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:07.984180 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:07.984182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:07.984328 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:07.984828 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:07.984837 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:07.985154 (+ 317us) client_negotiation.cc:769] Sending connection context
0811 05:16:07.985388 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 05:16:07.985811 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":133,"threads_started":1}
I20260811 05:16:07.988370 29890 heartbeater.cc:347] Connected to a master server at 127.7.67.190:38739
I20260811 05:16:07.988688 29890 heartbeater.cc:464] Registering TS with master...
I20260811 05:16:07.989358 29890 heartbeater.cc:511] Master 127.7.67.190:38739 requested a full tablet report, sending...
I20260811 05:16:07.991127 29579 ts_manager.cc:194] Registered new tserver with Master: bdabab5544a445619a9e3ceee6af4e6b (127.7.67.129:42341)
I20260811 05:16:07.992367 29579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.129@KRBTEST.COM'} at 127.7.67.129:56501
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
I20260811 05:16:08.004019 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:38739
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:38195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:16:08.127818 29897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:08.128093 29897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:08.128137 29897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:08.132128 29897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:08.132184 29897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:08.132247 29897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:16:08.137465 29897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:38195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:38739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:16:08.138520 29897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:08.139513 29897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:08.149276 29905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:08.149283 29906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:08.149293 29910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:08.149569 29897 server_base.cc:1034] running on GCE node
I20260811 05:16:08.150017 29897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:08.150714 29897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:08.151922 29897 hybrid_clock.cc:648] HybridClock initialized: now 1786425368151903 us; error 33 us; skew 500 ppm
Aug 11 05:16:08 dist-test-slave-04bb krb5kdc[29505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425368, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:08.154920 29897 init.cc:377] Logged in from keytab as kudu/127.7.67.130@KRBTEST.COM (short username kudu)
I20260811 05:16:08.155772 29897 webserver.cc:460] Webserver started at http://127.7.67.130:43285/ using document root <none> and password file <none>
I20260811 05:16:08.156343 29897 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:08.156404 29897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:08.156644 29897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:16:08.158700 29897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "b11eb4abed604862a99c16f06ff8f139"
format_stamp: "Formatted at 2026-08-11 05:16:08 on dist-test-slave-04bb"
server_key: "8622d41f7f1143006d7c046d6800b6a2"
server_key_iv: "91fd5ae090230deee00bb0d30dadf26c"
server_key_version: "encryptionkey@0"
I20260811 05:16:08.159228 29897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "b11eb4abed604862a99c16f06ff8f139"
format_stamp: "Formatted at 2026-08-11 05:16:08 on dist-test-slave-04bb"
server_key: "8622d41f7f1143006d7c046d6800b6a2"
server_key_iv: "91fd5ae090230deee00bb0d30dadf26c"
server_key_version: "encryptionkey@0"
I20260811 05:16:08.163009 29897 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 05:16:08.165511 29925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:08.166822 29897 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:16:08.166949 29897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "b11eb4abed604862a99c16f06ff8f139"
format_stamp: "Formatted at 2026-08-11 05:16:08 on dist-test-slave-04bb"
server_key: "8622d41f7f1143006d7c046d6800b6a2"
server_key_iv: "91fd5ae090230deee00bb0d30dadf26c"
server_key_version: "encryptionkey@0"
I20260811 05:16:08.167053 29897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:08.194208 29897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:08.197386 29897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:08.197595 29897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:08.198253 29897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:08.199236 29897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:16:08.199285 29897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:08.199328 29897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:16:08.199343 29897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:08.208776 29897 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:44347
I20260811 05:16:08.208796 30118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:44347 every 8 connection(s)
I20260811 05:16:08.209757 29897 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:16:08.212051 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 29897
I20260811 05:16:08.212141 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:16:08.212442 7438 external_mini_cluster.cc:1428] Setting key ac08fe35553b692a47562e47422a9c88
Aug 11 05:16:08 dist-test-slave-04bb krb5kdc[29505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425368, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.131@KRBTEST.COM; defaulting to no policy
I20260811 05:16:08.225827 29640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:08.211687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:59587 (local address 127.7.67.190:38739)
0811 05:16:08.211927 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:08.211933 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:08.212807 (+ 874us) server_negotiation.cc:407] Connection header received
0811 05:16:08.214315 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:08.214319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:08.214376 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:08.214477 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:08.215701 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.216632 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.218499 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.218770 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.218860 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:08.221995 (+ 3135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:08.221999 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:08.222001 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:08.222026 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:08.223628 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:08.224165 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:08.224169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:08.224171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:08.224259 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:08.224666 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:08.224669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:08.224670 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:08.224855 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:08.224974 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:08.225581 (+ 607us) server_negotiation.cc:299] Negotiation successful
0811 05:16:08.225694 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260811 05:16:08.226747 30121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:08.212068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.130:59587)
0811 05:16:08.212636 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:08.212693 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:08.214006 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:08.214640 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:08.214647 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:08.215014 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:08.215548 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:08.215561 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.216782 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.216788 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:08.218342 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:08.218357 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.218944 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.218963 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:08.219207 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:08.220001 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:08.220030 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:08.221816 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:08.223758 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:08.223764 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:08.223782 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:08.224073 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:08.224380 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:08.224389 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:08.224390 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:08.224575 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:08.225020 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:08.225030 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:08.225327 (+ 297us) client_negotiation.cc:769] Sending connection context
0811 05:16:08.225490 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 05:16:08.225762 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":136,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.131@KRBTEST.COM" created.
I20260811 05:16:08.228021 30119 heartbeater.cc:347] Connected to a master server at 127.7.67.190:38739
I20260811 05:16:08.228319 30119 heartbeater.cc:464] Registering TS with master...
I20260811 05:16:08.229007 30119 heartbeater.cc:511] Master 127.7.67.190:38739 requested a full tablet report, sending...
I20260811 05:16:08.230273 29579 ts_manager.cc:194] Registered new tserver with Master: b11eb4abed604862a99c16f06ff8f139 (127.7.67.130:44347)
I20260811 05:16:08.230939 29579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.130@KRBTEST.COM'} at 127.7.67.130:59587
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
I20260811 05:16:08.247210 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:38739
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:38195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:16:08.367162 30126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:08.367486 30126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:08.367532 30126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:08.371627 30126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:08.371695 30126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:08.371755 30126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:16:08.377063 30126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:38195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:38739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:16:08.378098 30126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:08.379089 30126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:08.389093 30138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:08.389053 30133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:08.389048 30135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:08.389400 30126 server_base.cc:1034] running on GCE node
I20260811 05:16:08.389899 30126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:08.390569 30126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:08.391762 30126 hybrid_clock.cc:648] HybridClock initialized: now 1786425368391727 us; error 46 us; skew 500 ppm
Aug 11 05:16:08 dist-test-slave-04bb krb5kdc[29505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425368, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:08.394623 30126 init.cc:377] Logged in from keytab as kudu/127.7.67.131@KRBTEST.COM (short username kudu)
I20260811 05:16:08.395550 30126 webserver.cc:460] Webserver started at http://127.7.67.131:41533/ using document root <none> and password file <none>
I20260811 05:16:08.396138 30126 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:08.396203 30126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:08.396466 30126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:16:08.398706 30126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "d4db37d43e274283827a6aba9506b74d"
format_stamp: "Formatted at 2026-08-11 05:16:08 on dist-test-slave-04bb"
server_key: "2810674ecc90ffeff2f86aee2926efdb"
server_key_iv: "c36eeb40584d0ae6356d46c109de9f1b"
server_key_version: "encryptionkey@0"
I20260811 05:16:08.399353 30126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "d4db37d43e274283827a6aba9506b74d"
format_stamp: "Formatted at 2026-08-11 05:16:08 on dist-test-slave-04bb"
server_key: "2810674ecc90ffeff2f86aee2926efdb"
server_key_iv: "c36eeb40584d0ae6356d46c109de9f1b"
server_key_version: "encryptionkey@0"
I20260811 05:16:08.403264 30126 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 05:16:08.405977 30148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:08.407285 30126 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:16:08.407411 30126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "d4db37d43e274283827a6aba9506b74d"
format_stamp: "Formatted at 2026-08-11 05:16:08 on dist-test-slave-04bb"
server_key: "2810674ecc90ffeff2f86aee2926efdb"
server_key_iv: "c36eeb40584d0ae6356d46c109de9f1b"
server_key_version: "encryptionkey@0"
I20260811 05:16:08.407512 30126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:08.419155 30126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:08.422611 30126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:08.422847 30126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:08.423493 30126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:08.424408 30126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:16:08.424458 30126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:08.424499 30126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:16:08.424602 30126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:08.433306 30126 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:35611
I20260811 05:16:08.433388 30349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:35611 every 8 connection(s)
I20260811 05:16:08.434269 30126 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:16:08.435149 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 30126
I20260811 05:16:08.435271 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:16:08.435600 7438 external_mini_cluster.cc:1428] Setting key 023a4d64e6bad5c5d8d240c4030cc5f1
Aug 11 05:16:08 dist-test-slave-04bb krb5kdc[29505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425368, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:16:08.449203 29640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:08.436484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:45189 (local address 127.7.67.190:38739)
0811 05:16:08.436676 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:08.436681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:08.437542 (+ 861us) server_negotiation.cc:407] Connection header received
0811 05:16:08.438961 (+ 1419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:08.438965 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:08.439022 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:08.439105 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:08.440301 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.440895 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.442073 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.442300 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.442409 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:08.445137 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:08.445142 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:08.445145 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:08.445180 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:08.446819 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:08.447402 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:08.447406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:08.447407 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:08.447496 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:08.447899 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:08.447901 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:08.447902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:08.448074 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:08.448150 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:08.448893 (+ 743us) server_negotiation.cc:299] Negotiation successful
0811 05:16:08.449023 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260811 05:16:08.450232 30355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:08.436861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.131:45189)
0811 05:16:08.437393 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:08.437437 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:08.438749 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:08.439276 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:08.439283 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:08.439642 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:08.440126 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:08.440140 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.441019 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.441023 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:08.441940 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:08.441950 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.442461 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.442465 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:08.442636 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:08.443228 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:08.443245 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:08.444938 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:08.446930 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:08.446935 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:08.446947 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:08.447286 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:08.447608 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:08.447612 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:08.447626 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:08.447797 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:08.448222 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:08.448229 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:08.448587 (+ 358us) client_negotiation.cc:769] Sending connection context
0811 05:16:08.448834 (+ 247us) client_negotiation.cc:240] Negotiation successful
0811 05:16:08.449206 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":162,"threads_started":1}
I20260811 05:16:08.451376 30350 heartbeater.cc:347] Connected to a master server at 127.7.67.190:38739
I20260811 05:16:08.451648 30350 heartbeater.cc:464] Registering TS with master...
I20260811 05:16:08.452164 30350 heartbeater.cc:511] Master 127.7.67.190:38739 requested a full tablet report, sending...
I20260811 05:16:08.453233 29579 ts_manager.cc:194] Registered new tserver with Master: d4db37d43e274283827a6aba9506b74d (127.7.67.131:35611)
I20260811 05:16:08.453768 29579 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.131@KRBTEST.COM'} at 127.7.67.131:45189
I20260811 05:16:08.462263 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:16:08.472074 29640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:08.464396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33130 (local address 127.7.67.190:38739)
0811 05:16:08.464559 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:08.464566 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:08.464727 (+ 161us) server_negotiation.cc:407] Connection header received
0811 05:16:08.464819 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:08.464821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:08.464860 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:08.464929 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:08.465699 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.466222 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.467030 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.467234 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.467315 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:08.468710 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:08.468713 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:08.468715 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:08.468747 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:08.470305 (+ 1558us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:08.470738 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:08.470742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:08.470744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:08.470827 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:08.471133 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:08.471137 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:08.471141 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:08.471420 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:08.471567 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:08.471805 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 05:16:08.471916 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 05:16:08.475070 29579 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33130:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:16:08.478433 29579 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:16:08.494086 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:08.487578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:42341 (local address 127.0.0.1:45974)
0811 05:16:08.488622 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:08.488636 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:08.488733 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:08.489515 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:08.489518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:08.489536 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:08.489649 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:08.489656 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.491098 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.491101 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:08.492405 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:08.492415 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.493516 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.493520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:08.493603 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:08.493636 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 05:16:08.493790 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 05:16:08.493909 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":975,"mutex_wait_us":259,"thread_start_us":123,"threads_started":1}
I20260811 05:16:08.494096 30374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:08.487683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45974 (local address 127.7.67.129:42341)
0811 05:16:08.489091 (+ 1408us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:08.489098 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:08.489120 (+ 22us) server_negotiation.cc:407] Connection header received
0811 05:16:08.489193 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:08.489197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:08.489371 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:08.489517 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:08.489752 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.490926 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.492571 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.493376 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.493503 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:08.493664 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:08.493830 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 05:16:08.493933 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1296,"thread_start_us":92,"threads_started":1}
I20260811 05:16:08.495568 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:08.487575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:44347 (local address 127.0.0.1:54926)
0811 05:16:08.488030 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:08.488079 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:08.488221 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:08.489384 (+ 1163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:08.489390 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:08.489417 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:08.489571 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:08.489579 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.491055 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.491058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:08.492673 (+ 1615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:08.492685 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.495158 (+ 2473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.495162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:08.495258 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:08.495279 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 05:16:08.495358 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 05:16:08.495431 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"mutex_wait_us":107,"thread_start_us":92,"threads_started":1}
I20260811 05:16:08.495594 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:08.488429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54926 (local address 127.7.67.130:44347)
0811 05:16:08.488925 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:08.488937 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:08.488971 (+ 34us) server_negotiation.cc:407] Connection header received
0811 05:16:08.489053 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:08.489060 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:08.489234 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:08.489359 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:08.489705 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.490926 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.493993 (+ 3067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.495003 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.495146 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:08.495278 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:08.495384 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 05:16:08.495466 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":141,"threads_started":1}
I20260811 05:16:08.495980 30370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:08.488312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:35611 (local address 127.0.0.1:57446)
0811 05:16:08.489587 (+ 1275us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:08.489602 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:08.489701 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:08.490110 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:08.490118 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:08.490135 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:08.490250 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:08.490264 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.491309 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.491313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:08.492674 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:08.492685 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.493792 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.493796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:08.493892 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:08.493914 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 05:16:08.493991 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 05:16:08.494045 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1218,"mutex_wait_us":119,"thread_start_us":106,"threads_started":1}
I20260811 05:16:08.495612 30380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:08.488620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57446 (local address 127.7.67.131:35611)
0811 05:16:08.489028 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:08.489039 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:08.489797 (+ 758us) server_negotiation.cc:407] Connection header received
0811 05:16:08.489863 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:08.489866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:08.489986 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:08.490136 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:08.490392 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.491159 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.492808 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.493654 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.493793 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:08.493930 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:08.495367 (+ 1437us) server_negotiation.cc:299] Negotiation successful
0811 05:16:08.495473 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":137,"threads_started":1}
I20260811 05:16:08.497452 29777 tablet_service.cc:1467] Processing CreateTablet for tablet 5c1fbf549ebc483eb39a8cabc2445387 (DEFAULT_TABLE table=test-table [id=4787e8c4d4c94dae874f8ce8db1000f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:16:08.499078 29777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c1fbf549ebc483eb39a8cabc2445387. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:08.499055 30232 tablet_service.cc:1467] Processing CreateTablet for tablet 5c1fbf549ebc483eb39a8cabc2445387 (DEFAULT_TABLE table=test-table [id=4787e8c4d4c94dae874f8ce8db1000f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:16:08.499287 30009 tablet_service.cc:1467] Processing CreateTablet for tablet 5c1fbf549ebc483eb39a8cabc2445387 (DEFAULT_TABLE table=test-table [id=4787e8c4d4c94dae874f8ce8db1000f4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:16:08.500118 30232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c1fbf549ebc483eb39a8cabc2445387. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:08.500207 30009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c1fbf549ebc483eb39a8cabc2445387. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:08.506079 30382 tablet_bootstrap.cc:492] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139: Bootstrap starting.
I20260811 05:16:08.507798 30383 tablet_bootstrap.cc:492] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b: Bootstrap starting.
I20260811 05:16:08.508908 30384 tablet_bootstrap.cc:492] T 5c1fbf549ebc483eb39a8cabc2445387 P d4db37d43e274283827a6aba9506b74d: Bootstrap starting.
I20260811 05:16:08.509102 30382 tablet_bootstrap.cc:654] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:08.509985 30382 log.cc:824] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:08.510237 30383 tablet_bootstrap.cc:654] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:08.511049 30383 log.cc:824] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:08.511310 30384 tablet_bootstrap.cc:654] T 5c1fbf549ebc483eb39a8cabc2445387 P d4db37d43e274283827a6aba9506b74d: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:08.511968 30384 log.cc:824] T 5c1fbf549ebc483eb39a8cabc2445387 P d4db37d43e274283827a6aba9506b74d: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:08.512127 30382 tablet_bootstrap.cc:492] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139: No bootstrap required, opened a new log
I20260811 05:16:08.512351 30382 ts_tablet_manager.cc:1397] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:16:08.513739 30383 tablet_bootstrap.cc:492] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b: No bootstrap required, opened a new log
I20260811 05:16:08.513944 30383 ts_tablet_manager.cc:1397] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 05:16:08.514029 30384 tablet_bootstrap.cc:492] T 5c1fbf549ebc483eb39a8cabc2445387 P d4db37d43e274283827a6aba9506b74d: No bootstrap required, opened a new log
I20260811 05:16:08.514226 30384 ts_tablet_manager.cc:1397] T 5c1fbf549ebc483eb39a8cabc2445387 P d4db37d43e274283827a6aba9506b74d: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:16:08.516429 30382 raft_consensus.cc:348] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } }
I20260811 05:16:08.516757 30382 raft_consensus.cc:374] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:08.516821 30382 raft_consensus.cc:729] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b11eb4abed604862a99c16f06ff8f139, State: Initialized, Role: FOLLOWER
I20260811 05:16:08.517323 30382 consensus_queue.cc:260] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } }
I20260811 05:16:08.517925 30383 raft_consensus.cc:348] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } }
I20260811 05:16:08.518155 30119 heartbeater.cc:503] Master 127.7.67.190:38739 was elected leader, sending a full tablet report...
I20260811 05:16:08.518168 30383 raft_consensus.cc:374] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:08.518229 30383 raft_consensus.cc:729] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdabab5544a445619a9e3ceee6af4e6b, State: Initialized, Role: FOLLOWER
I20260811 05:16:08.518059 30384 raft_consensus.cc:348] T 5c1fbf549ebc483eb39a8cabc2445387 P d4db37d43e274283827a6aba9506b74d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } }
I20260811 05:16:08.518309 30384 raft_consensus.cc:374] T 5c1fbf549ebc483eb39a8cabc2445387 P d4db37d43e274283827a6aba9506b74d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:08.518360 30384 raft_consensus.cc:729] T 5c1fbf549ebc483eb39a8cabc2445387 P d4db37d43e274283827a6aba9506b74d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4db37d43e274283827a6aba9506b74d, State: Initialized, Role: FOLLOWER
I20260811 05:16:08.518440 30382 ts_tablet_manager.cc:1428] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139: Time spent starting tablet: real 0.006s user 0.001s sys 0.004s
I20260811 05:16:08.518674 30383 consensus_queue.cc:260] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } }
I20260811 05:16:08.518785 30384 consensus_queue.cc:260] T 5c1fbf549ebc483eb39a8cabc2445387 P d4db37d43e274283827a6aba9506b74d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } }
I20260811 05:16:08.519430 29890 heartbeater.cc:503] Master 127.7.67.190:38739 was elected leader, sending a full tablet report...
I20260811 05:16:08.519630 30383 ts_tablet_manager.cc:1428] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 05:16:08.519781 30384 ts_tablet_manager.cc:1428] T 5c1fbf549ebc483eb39a8cabc2445387 P d4db37d43e274283827a6aba9506b74d: Time spent starting tablet: real 0.005s user 0.000s sys 0.006s
I20260811 05:16:08.520128 30350 heartbeater.cc:503] Master 127.7.67.190:38739 was elected leader, sending a full tablet report...
I20260811 05:16:08.589829 30389 raft_consensus.cc:482] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:16:08.590080 30389 raft_consensus.cc:504] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } }
I20260811 05:16:08.591312 30389 leader_election.cc:288] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bdabab5544a445619a9e3ceee6af4e6b (127.7.67.129:42341), d4db37d43e274283827a6aba9506b74d (127.7.67.131:35611)
I20260811 05:16:08.596300 30380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:08.591838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:33849 (local address 127.7.67.131:35611)
0811 05:16:08.591966 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:08.591971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:08.592213 (+ 242us) server_negotiation.cc:407] Connection header received
0811 05:16:08.592302 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:08.592304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:08.592365 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:08.592445 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:08.592694 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.593416 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.594873 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.595563 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.595739 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:08.595800 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:08.596073 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 05:16:08.596154 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 05:16:08.596300 30374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:08.591674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:53939 (local address 127.7.67.129:42341)
0811 05:16:08.591840 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:08.591845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:08.591862 (+ 17us) server_negotiation.cc:407] Connection header received
0811 05:16:08.591931 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:08.591937 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:08.591998 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:08.592068 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:08.592410 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.593237 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.594873 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.595617 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.595767 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:08.595817 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:08.596073 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 05:16:08.596154 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 05:16:08.596321 30121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:08.591498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:42341 (local address 127.7.67.130:53939)
0811 05:16:08.591648 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:08.591677 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:08.591790 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:08.592107 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:08.592112 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:08.592139 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:08.592282 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:08.592290 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.593379 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.593382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:08.594699 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:08.594712 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.595729 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.595732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:08.595907 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:08.595955 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 05:16:08.596034 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 05:16:08.596166 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"mutex_wait_us":93}
I20260811 05:16:08.596330 30395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:08.591624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:35611 (local address 127.7.67.130:33849)
0811 05:16:08.592105 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:08.592119 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:08.592218 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:08.592461 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:08.592463 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:08.592478 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:08.592585 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:08.592589 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.593522 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.593525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:08.594718 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:08.594725 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.595722 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.595725 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:08.595903 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:08.595950 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 05:16:08.596034 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 05:16:08.596163 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":96,"thread_start_us":172,"threads_started":1}
I20260811 05:16:08.597168 29812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c1fbf549ebc483eb39a8cabc2445387" candidate_uuid: "b11eb4abed604862a99c16f06ff8f139" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdabab5544a445619a9e3ceee6af4e6b" is_pre_election: true
I20260811 05:16:08.597168 30270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c1fbf549ebc483eb39a8cabc2445387" candidate_uuid: "b11eb4abed604862a99c16f06ff8f139" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4db37d43e274283827a6aba9506b74d" is_pre_election: true
I20260811 05:16:08.597484 29812 raft_consensus.cc:2454] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b11eb4abed604862a99c16f06ff8f139 in term 0.
I20260811 05:16:08.597483 30270 raft_consensus.cc:2454] T 5c1fbf549ebc483eb39a8cabc2445387 P d4db37d43e274283827a6aba9506b74d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b11eb4abed604862a99c16f06ff8f139 in term 0.
I20260811 05:16:08.598063 29930 leader_election.cc:302] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: b11eb4abed604862a99c16f06ff8f139, bdabab5544a445619a9e3ceee6af4e6b, d4db37d43e274283827a6aba9506b74d; no voters:
I20260811 05:16:08.598377 30389 raft_consensus.cc:2781] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:16:08.598467 30389 raft_consensus.cc:482] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:16:08.598515 30389 raft_consensus.cc:3037] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:08.599736 30389 raft_consensus.cc:504] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } }
I20260811 05:16:08.600078 30389 leader_election.cc:288] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139 [CANDIDATE]: Term 1 election: Requested vote from peers bdabab5544a445619a9e3ceee6af4e6b (127.7.67.129:42341), d4db37d43e274283827a6aba9506b74d (127.7.67.131:35611)
I20260811 05:16:08.600447 29812 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c1fbf549ebc483eb39a8cabc2445387" candidate_uuid: "b11eb4abed604862a99c16f06ff8f139" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bdabab5544a445619a9e3ceee6af4e6b"
I20260811 05:16:08.600452 30270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c1fbf549ebc483eb39a8cabc2445387" candidate_uuid: "b11eb4abed604862a99c16f06ff8f139" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4db37d43e274283827a6aba9506b74d"
I20260811 05:16:08.600607 30270 raft_consensus.cc:3037] T 5c1fbf549ebc483eb39a8cabc2445387 P d4db37d43e274283827a6aba9506b74d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:08.600621 29812 raft_consensus.cc:3037] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:08.601620 30270 raft_consensus.cc:2454] T 5c1fbf549ebc483eb39a8cabc2445387 P d4db37d43e274283827a6aba9506b74d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b11eb4abed604862a99c16f06ff8f139 in term 1.
I20260811 05:16:08.601769 29812 raft_consensus.cc:2454] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b11eb4abed604862a99c16f06ff8f139 in term 1.
I20260811 05:16:08.601929 29930 leader_election.cc:302] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b11eb4abed604862a99c16f06ff8f139, d4db37d43e274283827a6aba9506b74d; no voters:
I20260811 05:16:08.602103 30389 raft_consensus.cc:2781] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:16:08.602357 30389 raft_consensus.cc:686] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139 [term 1 LEADER]: Becoming Leader. State: Replica: b11eb4abed604862a99c16f06ff8f139, State: Running, Role: LEADER
I20260811 05:16:08.602705 30389 consensus_queue.cc:237] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } }
I20260811 05:16:08.606140 29578 catalog_manager.cc:5495] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139 reported cstate change: term changed from 0 to 1, leader changed from <none> to b11eb4abed604862a99c16f06ff8f139 (127.7.67.130). New cstate: current_term: 1 leader_uuid: "b11eb4abed604862a99c16f06ff8f139" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } health_report { overall_health: UNKNOWN } } }
I20260811 05:16:08.647532 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:08.643724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54942 (local address 127.7.67.130:44347)
0811 05:16:08.643882 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:08.643885 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:08.643902 (+ 17us) server_negotiation.cc:407] Connection header received
0811 05:16:08.643980 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:08.643983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:08.644041 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:08.644147 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:16:08.644431 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.644999 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.645907 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:08.646118 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:08.646211 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:08.646591 (+ 380us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:16:08.647050 (+ 459us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:16:08.647204 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:08.647269 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 05:16:08.647324 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 05:16:08.653756 29812 raft_consensus.cc:1261] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b [term 1 FOLLOWER]: Refusing update from remote peer b11eb4abed604862a99c16f06ff8f139: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:16:08.653761 30270 raft_consensus.cc:1261] T 5c1fbf549ebc483eb39a8cabc2445387 P d4db37d43e274283827a6aba9506b74d [term 1 FOLLOWER]: Refusing update from remote peer b11eb4abed604862a99c16f06ff8f139: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:16:08.654520 30398 consensus_queue.cc:1035] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:16:08.654701 30389 consensus_queue.cc:1035] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:16:08.661803 30402 mvcc.cc:205] Tried to move back new op lower bound from 7317198309998161920 to 7317198309799399424. Current Snapshot: MvccSnapshot[applied={T|T < 7317198309998161920}]
I20260811 05:16:08.669322 30403 mvcc.cc:205] Tried to move back new op lower bound from 7317198309998161920 to 7317198309799399424. Current Snapshot: MvccSnapshot[applied={T|T < 7317198309998161920}]
I20260811 05:16:08.674235 29578 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:16:08.674407 29578 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:16:08.676783 29578 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e: Sending DeleteTablet for 3 replicas of tablet 5c1fbf549ebc483eb39a8cabc2445387
I20260811 05:16:08.677560 30009 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c1fbf549ebc483eb39a8cabc2445387 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:16:08 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:54926
I20260811 05:16:08.677567 29777 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c1fbf549ebc483eb39a8cabc2445387 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:16:08 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:45974
I20260811 05:16:08.677896 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 29681
I20260811 05:16:08.678076 30232 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c1fbf549ebc483eb39a8cabc2445387 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:16:08 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:57446
I20260811 05:16:08.687671 30421 tablet_replica.cc:330] T 5c1fbf549ebc483eb39a8cabc2445387 P d4db37d43e274283827a6aba9506b74d: stopping tablet replica
I20260811 05:16:08.688048 30421 raft_consensus.cc:2229] T 5c1fbf549ebc483eb39a8cabc2445387 P d4db37d43e274283827a6aba9506b74d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:16:08.688401 30421 raft_consensus.cc:2258] T 5c1fbf549ebc483eb39a8cabc2445387 P d4db37d43e274283827a6aba9506b74d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:16:08.690979 30421 ts_tablet_manager.cc:1905] T 5c1fbf549ebc483eb39a8cabc2445387 P d4db37d43e274283827a6aba9506b74d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:16:08.694628 30421 ts_tablet_manager.cc:1918] T 5c1fbf549ebc483eb39a8cabc2445387 P d4db37d43e274283827a6aba9506b74d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:16:08.694743 30421 log.cc:1196] T 5c1fbf549ebc483eb39a8cabc2445387 P d4db37d43e274283827a6aba9506b74d: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/5c1fbf549ebc483eb39a8cabc2445387
I20260811 05:16:08.695070 30419 tablet_replica.cc:330] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139: stopping tablet replica
I20260811 05:16:08.695091 30421 ts_tablet_manager.cc:1939] T 5c1fbf549ebc483eb39a8cabc2445387 P d4db37d43e274283827a6aba9506b74d: Deleting consensus metadata
I20260811 05:16:08.695494 30419 raft_consensus.cc:2229] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:16:08.695998 30419 raft_consensus.cc:2258] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:16:08.696638 29559 catalog_manager.cc:4841] TS d4db37d43e274283827a6aba9506b74d (127.7.67.131:35611): tablet 5c1fbf549ebc483eb39a8cabc2445387 (table test-table [id=4787e8c4d4c94dae874f8ce8db1000f4]) successfully deleted
I20260811 05:16:08.697751 30419 ts_tablet_manager.cc:1905] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 05:16:08.699281 29560 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.129:42341 (error 108)
I20260811 05:16:08.699581 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 29897
I20260811 05:16:08.706389 30370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:08.706026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:42341 (local address 127.0.0.1:45982)
0811 05:16:08.706213 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:08.706303 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.129:42341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260811 05:16:08.706691 29560 catalog_manager.cc:4584] TS bdabab5544a445619a9e3ceee6af4e6b (127.7.67.129:42341): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5c1fbf549ebc483eb39a8cabc2445387: Network error: Client connection negotiation failed: client connection to 127.7.67.129:42341: connect: Connection refused (error 111)
I20260811 05:16:08.712374 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 30126
I20260811 05:16:08.712610 30370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:08.712269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:44347 (local address 127.0.0.1:54952)
0811 05:16:08.712411 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:08.712480 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.130:44347: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 05:16:08.728609 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 29527
I20260811 05:16:08.744144 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:38739
--webserver_interface=127.7.67.190
--webserver_port=33627
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:38195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:38739
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:16:08.864101 30424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:08.864445 30424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:08.864591 30424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:08.868749 30424 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:16:08.868886 30424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:08.868932 30424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:08.868973 30424 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:16:08.869014 30424 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:16:08.874670 30424 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:38195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:38739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:38739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=33627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30424
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:16:08.875885 30424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:08.877049 30424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:08.891902 30433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:08.891882 30437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:08.891882 30432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:08.892206 30424 server_base.cc:1034] running on GCE node
I20260811 05:16:08.893082 30424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:08.894367 30424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:08.895565 30424 hybrid_clock.cc:648] HybridClock initialized: now 1786425368895548 us; error 37 us; skew 500 ppm
Aug 11 05:16:08 dist-test-slave-04bb krb5kdc[29505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425368, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:08.898662 30424 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:16:08.899641 30424 webserver.cc:460] Webserver started at http://127.7.67.190:33627/ using document root <none> and password file <none>
I20260811 05:16:08.900298 30424 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:08.900424 30424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:08.904726 30424 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:16:08.907208 30457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:08.908624 30424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:16:08.908767 30424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "0c8a1f31a2f9487ab869de998b28f97e"
format_stamp: "Formatted at 2026-08-11 05:16:07 on dist-test-slave-04bb"
server_key: "b85f448a35f1f34828dc85dea019e4fb"
server_key_iv: "cda249c7ad42af3d947ec61529234837"
server_key_version: "encryptionkey@0"
I20260811 05:16:08.909426 30424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:08.933123 30424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:08.937142 30424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:08.937398 30424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:08.945255 30424 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:38739
I20260811 05:16:08.945566 30543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:38739 every 8 connection(s)
I20260811 05:16:08.946465 30424 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:16:08.951174 30544 sys_catalog.cc:263] Verifying existing consensus state
I20260811 05:16:08.951987 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 30424
I20260811 05:16:08.952550 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:42341
--local_ip_for_outbound_sockets=127.7.67.129
--tserver_master_addrs=127.7.67.190:38739
--webserver_port=42163
--webserver_interface=127.7.67.129
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:38195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:16:08.953436 30544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e: Bootstrap starting.
I20260811 05:16:08.971966 30544 log.cc:824] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:08.981308 30544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:16:08.981735 30544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e: Bootstrap complete.
I20260811 05:16:08.986007 30544 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8a1f31a2f9487ab869de998b28f97e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38739 } }
I20260811 05:16:08.986428 30544 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c8a1f31a2f9487ab869de998b28f97e, State: Initialized, Role: FOLLOWER
I20260811 05:16:08.986935 30544 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8a1f31a2f9487ab869de998b28f97e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38739 } }
I20260811 05:16:08.987083 30544 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:16:08.987178 30544 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:16:08.987289 30544 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 1 FOLLOWER]: Advancing to term 2
I20260811 05:16:08.989382 30544 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8a1f31a2f9487ab869de998b28f97e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38739 } }
I20260811 05:16:08.989730 30544 leader_election.cc:302] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c8a1f31a2f9487ab869de998b28f97e; no voters:
I20260811 05:16:08.990103 30544 leader_election.cc:288] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 05:16:08.990192 30550 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 2 FOLLOWER]: Leader election won for term 2
I20260811 05:16:08.990425 30550 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 2 LEADER]: Becoming Leader. State: Replica: 0c8a1f31a2f9487ab869de998b28f97e, State: Running, Role: LEADER
I20260811 05:16:08.990732 30550 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8a1f31a2f9487ab869de998b28f97e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38739 } }
I20260811 05:16:08.991417 30544 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:16:08.991757 30552 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c8a1f31a2f9487ab869de998b28f97e. Latest consensus state: current_term: 2 leader_uuid: "0c8a1f31a2f9487ab869de998b28f97e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8a1f31a2f9487ab869de998b28f97e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38739 } } }
I20260811 05:16:08.991895 30552 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [sys.catalog]: This master's current role is: LEADER
I20260811 05:16:08.992137 30551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0c8a1f31a2f9487ab869de998b28f97e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8a1f31a2f9487ab869de998b28f97e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38739 } } }
I20260811 05:16:08.992250 30551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [sys.catalog]: This master's current role is: LEADER
I20260811 05:16:08.992661 30556 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:16:08.997838 30556 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:16:08.998848 30556 catalog_manager.cc:1259] Loaded cluster ID: 49ebe35220b74d2f905142fc77f917f8
I20260811 05:16:08.998917 30556 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:16:09.000396 30556 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 05:16:09.005312 30556 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 05:16:09.010908 30556 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:16:09.011708 30556 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e: Loaded TSK: 0
I20260811 05:16:09.012964 30556 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 05:16:09.075790 30546 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:09.076084 30546 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:09.076175 30546 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:09.080259 30546 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:09.080349 30546 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:09.080435 30546 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:16:09.085772 30546 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:38195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:42341
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=42163
--tserver_master_addrs=127.7.67.190:38739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30546
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:16:09.086838 30546 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:09.088014 30546 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:09.098239 30597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:09.098284 30592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:09.098357 30591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:09.098618 30546 server_base.cc:1034] running on GCE node
I20260811 05:16:09.099129 30546 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:09.099857 30546 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:09.101032 30546 hybrid_clock.cc:648] HybridClock initialized: now 1786425369101012 us; error 39 us; skew 500 ppm
Aug 11 05:16:09 dist-test-slave-04bb krb5kdc[29505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425369, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:09.103909 30546 init.cc:377] Logged in from keytab as kudu/127.7.67.129@KRBTEST.COM (short username kudu)
I20260811 05:16:09.104967 30546 webserver.cc:460] Webserver started at http://127.7.67.129:42163/ using document root <none> and password file <none>
I20260811 05:16:09.105542 30546 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:09.105618 30546 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:09.109745 30546 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:16:09.112016 30607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:09.113416 30546 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 05:16:09.113513 30546 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "bdabab5544a445619a9e3ceee6af4e6b"
format_stamp: "Formatted at 2026-08-11 05:16:07 on dist-test-slave-04bb"
server_key: "342143cf6cd5e69411c1f1b3e3297ea0"
server_key_iv: "f9dcdc70efd59352e1170b5035a9a2fc"
server_key_version: "encryptionkey@0"
I20260811 05:16:09.114100 30546 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:09.132092 30546 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:09.135601 30546 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:09.135819 30546 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:09.136511 30546 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:09.137706 30618 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 05:16:09.142079 30546 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 05:16:09.142144 30546 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.001s
I20260811 05:16:09.142217 30546 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 05:16:09.144090 30546 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 05:16:09.144150 30546 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 05:16:09.144434 30618 tablet_bootstrap.cc:492] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b: Bootstrap starting.
I20260811 05:16:09.155331 30546 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:42341
I20260811 05:16:09.155344 30806 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:42341 every 8 connection(s)
I20260811 05:16:09.156642 30546 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:16:09.160969 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 30546
I20260811 05:16:09.161490 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:44347
--local_ip_for_outbound_sockets=127.7.67.130
--tserver_master_addrs=127.7.67.190:38739
--webserver_port=43285
--webserver_interface=127.7.67.130
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:38195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:16:09.166392 30618 log.cc:824] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b: Log is configured to *not* fsync() on all Append() calls
Aug 11 05:16:09 dist-test-slave-04bb krb5kdc[29505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425369, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:16:09.175326 30618 tablet_bootstrap.cc:492] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:16:09.175745 30618 tablet_bootstrap.cc:492] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b: Bootstrap complete.
I20260811 05:16:09.176942 30618 ts_tablet_manager.cc:1397] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b: Time spent bootstrapping tablet: real 0.033s user 0.013s sys 0.018s
I20260811 05:16:09.177341 30811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:09.158934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.129:58545)
0811 05:16:09.159540 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:09.159582 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:09.160435 (+ 853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:09.161915 (+ 1480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:09.161923 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:09.162348 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:09.164127 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:09.164142 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.165559 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.165562 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:09.166432 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:09.166443 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.167390 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.167397 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:09.167639 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:09.168435 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:09.168463 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:09.170599 (+ 2136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:09.173416 (+ 2817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:09.173425 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:09.173444 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:09.173851 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:09.174279 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:09.174285 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:09.174289 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:09.174518 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:09.175255 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:09.175266 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:09.175629 (+ 363us) client_negotiation.cc:769] Sending connection context
0811 05:16:09.175862 (+ 233us) client_negotiation.cc:240] Negotiation successful
0811 05:16:09.176259 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":144,"threads_started":1}
I20260811 05:16:09.179229 30810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:09.158902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:58545 (local address 127.7.67.190:38739)
0811 05:16:09.159270 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:09.159287 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:09.159707 (+ 420us) server_negotiation.cc:407] Connection header received
0811 05:16:09.160917 (+ 1210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:09.160928 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:09.161297 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:09.161646 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:09.164332 (+ 2686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.165389 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.166577 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.167205 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.167405 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:09.170845 (+ 3440us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:09.170852 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:09.170870 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:09.170905 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:09.173273 (+ 2368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:09.173991 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:09.173997 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:09.174003 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:09.174165 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:09.174639 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:09.174643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:09.174644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:09.175119 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:09.175272 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:09.178088 (+ 2816us) server_negotiation.cc:299] Negotiation successful
0811 05:16:09.178410 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":125,"threads_started":1}
I20260811 05:16:09.179518 30618 raft_consensus.cc:348] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } }
I20260811 05:16:09.180063 30618 raft_consensus.cc:729] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdabab5544a445619a9e3ceee6af4e6b, State: Initialized, Role: FOLLOWER
I20260811 05:16:09.180591 30618 consensus_queue.cc:260] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } }
I20260811 05:16:09.181542 30618 ts_tablet_manager.cc:1428] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:16:09.181993 30807 heartbeater.cc:347] Connected to a master server at 127.7.67.190:38739
I20260811 05:16:09.182238 30807 heartbeater.cc:464] Registering TS with master...
I20260811 05:16:09.182797 30807 heartbeater.cc:511] Master 127.7.67.190:38739 requested a full tablet report, sending...
I20260811 05:16:09.185468 30481 ts_manager.cc:194] Registered new tserver with Master: bdabab5544a445619a9e3ceee6af4e6b (127.7.67.129:42341)
I20260811 05:16:09.189505 30481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.129@KRBTEST.COM'} at 127.7.67.129:58545
I20260811 05:16:09.192394 30807 heartbeater.cc:503] Master 127.7.67.190:38739 was elected leader, sending a full tablet report...
Aug 11 05:16:09 dist-test-slave-04bb krb5kdc[29505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425368, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for kudu/127.7.67.129@KRBTEST.COM
I20260811 05:16:09.199654 30817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:09.188286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:42341 (local address 127.0.0.1:45996)
0811 05:16:09.189130 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:09.189177 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:09.189343 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:09.189916 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:09.189922 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:09.190306 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:09.190459 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:09.190467 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.191791 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.191795 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:09.192875 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:09.192888 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.193649 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.193653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:09.193771 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:09.194380 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:09.194392 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:09.195781 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:09.197907 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:09.197915 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:09.197920 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:09.198152 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:09.198472 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:09.198474 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:09.198476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:09.198554 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:09.199057 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:09.199074 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:09.199219 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 05:16:09.199352 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 05:16:09.199514 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":127,"threads_started":1}
I20260811 05:16:09.199785 30823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:09.188388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45996 (local address 127.7.67.129:42341)
0811 05:16:09.188830 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:09.188847 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:09.189293 (+ 446us) server_negotiation.cc:407] Connection header received
0811 05:16:09.189504 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:09.189509 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:09.189761 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:09.189911 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:09.190578 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.191673 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.193015 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.193510 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.193628 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:09.195931 (+ 2303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:09.195941 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:09.195954 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:09.195984 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:09.197796 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:09.198263 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:09.198269 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:09.198270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:09.198377 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:09.198640 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:09.198643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:09.198645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:09.198940 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:09.199075 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:09.199457 (+ 382us) server_negotiation.cc:299] Negotiation successful
0811 05:16:09.199628 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":148,"threads_started":1}
I20260811 05:16:09.201043 30687 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c1fbf549ebc483eb39a8cabc2445387 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:16:08 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:45996
I20260811 05:16:09.201912 30827 tablet_replica.cc:330] stopping tablet replica
I20260811 05:16:09.202214 30827 raft_consensus.cc:2229] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:16:09.202482 30827 raft_consensus.cc:2258] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:16:09.203478 30827 ts_tablet_manager.cc:1905] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:16:09.207212 30827 ts_tablet_manager.cc:1918] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:16:09.207299 30827 log.cc:1196] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/wal/wals/5c1fbf549ebc483eb39a8cabc2445387
I20260811 05:16:09.207571 30827 ts_tablet_manager.cc:1939] T 5c1fbf549ebc483eb39a8cabc2445387 P bdabab5544a445619a9e3ceee6af4e6b: Deleting consensus metadata
I20260811 05:16:09.209438 30460 catalog_manager.cc:4841] TS bdabab5544a445619a9e3ceee6af4e6b (127.7.67.129:42341): tablet 5c1fbf549ebc483eb39a8cabc2445387 (table test-table [id=4787e8c4d4c94dae874f8ce8db1000f4]) successfully deleted
W20260811 05:16:09.288638 30814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:09.288906 30814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:09.288950 30814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:09.293159 30814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:09.293212 30814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:09.293315 30814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:16:09.298578 30814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:38195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:44347
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=43285
--tserver_master_addrs=127.7.67.190:38739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:16:09.299731 30814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:09.300793 30814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:09.310768 30837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:09.310802 30839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:09.310768 30842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:09.311144 30814 server_base.cc:1034] running on GCE node
I20260811 05:16:09.311606 30814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:09.312251 30814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:09.313391 30814 hybrid_clock.cc:648] HybridClock initialized: now 1786425369313346 us; error 53 us; skew 500 ppm
Aug 11 05:16:09 dist-test-slave-04bb krb5kdc[29505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425369, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:09.316246 30814 init.cc:377] Logged in from keytab as kudu/127.7.67.130@KRBTEST.COM (short username kudu)
I20260811 05:16:09.317224 30814 webserver.cc:460] Webserver started at http://127.7.67.130:43285/ using document root <none> and password file <none>
I20260811 05:16:09.317862 30814 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:09.317961 30814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:09.322006 30814 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:16:09.324349 30855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:09.325671 30814 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:16:09.325806 30814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "b11eb4abed604862a99c16f06ff8f139"
format_stamp: "Formatted at 2026-08-11 05:16:08 on dist-test-slave-04bb"
server_key: "8622d41f7f1143006d7c046d6800b6a2"
server_key_iv: "91fd5ae090230deee00bb0d30dadf26c"
server_key_version: "encryptionkey@0"
I20260811 05:16:09.326372 30814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:09.354909 30814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:09.358438 30814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:09.358662 30814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:09.359313 30814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:09.360517 30865 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260811 05:16:09.364797 30865 ts_tablet_manager.cc:1852] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260811 05:16:09.365180 30865 ts_tablet_manager.cc:1905] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:16:09.369117 30865 ts_tablet_manager.cc:1918] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:16:09.369220 30865 log.cc:1196] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/5c1fbf549ebc483eb39a8cabc2445387
I20260811 05:16:09.369613 30865 ts_tablet_manager.cc:1939] T 5c1fbf549ebc483eb39a8cabc2445387 P b11eb4abed604862a99c16f06ff8f139: Deleting consensus metadata
I20260811 05:16:09.369951 30814 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260811 05:16:09.370038 30814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.010s user 0.000s sys 0.000s
I20260811 05:16:09.370123 30814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:16:09.370170 30814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:09.379467 30814 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:44347
I20260811 05:16:09.379503 31054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:44347 every 8 connection(s)
I20260811 05:16:09.380386 30814 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:16:09.382426 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 30814
I20260811 05:16:09.382916 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:35611
--local_ip_for_outbound_sockets=127.7.67.131
--tserver_master_addrs=127.7.67.190:38739
--webserver_port=41533
--webserver_interface=127.7.67.131
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:38195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:16:09 dist-test-slave-04bb krb5kdc[29505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425369, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:16:09.395802 30810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:09.382309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:43917 (local address 127.7.67.190:38739)
0811 05:16:09.382456 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:09.382460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:09.383585 (+ 1125us) server_negotiation.cc:407] Connection header received
0811 05:16:09.384558 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:09.384562 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:09.384626 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:09.384746 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:09.385940 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.386507 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.387648 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.387899 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.388013 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:09.391051 (+ 3038us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:09.391077 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:09.391083 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:09.391124 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:09.393153 (+ 2029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:09.393798 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:09.393804 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:09.393809 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:09.393926 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:09.394355 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:09.394360 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:09.394365 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:09.394657 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:09.394779 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:09.395378 (+ 599us) server_negotiation.cc:299] Negotiation successful
0811 05:16:09.395552 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 05:16:09.396682 31058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:09.382820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.130:43917)
0811 05:16:09.383435 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:09.383477 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:09.384326 (+ 849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:09.384928 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:09.384934 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:09.385288 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:09.385797 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:09.385810 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.386631 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.386639 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:09.387502 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:09.387512 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.388070 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.388074 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:09.388247 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:09.388965 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:09.388982 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:09.390849 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:09.393316 (+ 2467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:09.393322 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:09.393345 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:09.393696 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:09.394050 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:09.394053 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:09.394055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:09.394251 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:09.394798 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:09.394804 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:09.395070 (+ 266us) client_negotiation.cc:769] Sending connection context
0811 05:16:09.395294 (+ 224us) client_negotiation.cc:240] Negotiation successful
0811 05:16:09.395580 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":147,"threads_started":1}
I20260811 05:16:09.398056 31055 heartbeater.cc:347] Connected to a master server at 127.7.67.190:38739
I20260811 05:16:09.398347 31055 heartbeater.cc:464] Registering TS with master...
I20260811 05:16:09.398890 31055 heartbeater.cc:511] Master 127.7.67.190:38739 requested a full tablet report, sending...
I20260811 05:16:09.400092 30481 ts_manager.cc:194] Registered new tserver with Master: b11eb4abed604862a99c16f06ff8f139 (127.7.67.130:44347)
I20260811 05:16:09.400837 30481 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.130@KRBTEST.COM'} at 127.7.67.130:43917
W20260811 05:16:09.503115 31059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:09.503402 31059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:09.503489 31059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:09.507593 31059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:09.507676 31059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:09.507777 31059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:16:09.513197 31059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:38195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:35611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=41533
--tserver_master_addrs=127.7.67.190:38739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:16:09.514428 31059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:09.515452 31059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:09.525890 31067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:09.525897 31070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:09.526117 31059 server_base.cc:1034] running on GCE node
W20260811 05:16:09.526153 31072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:09.526645 31059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:09.527311 31059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:09.528512 31059 hybrid_clock.cc:648] HybridClock initialized: now 1786425369528494 us; error 35 us; skew 500 ppm
Aug 11 05:16:09 dist-test-slave-04bb krb5kdc[29505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425369, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:09.531579 31059 init.cc:377] Logged in from keytab as kudu/127.7.67.131@KRBTEST.COM (short username kudu)
I20260811 05:16:09.532474 31059 webserver.cc:460] Webserver started at http://127.7.67.131:41533/ using document root <none> and password file <none>
I20260811 05:16:09.533077 31059 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:09.533196 31059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:09.537123 31059 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:16:09.539149 31083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:09.540350 31059 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:16:09.540489 31059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "d4db37d43e274283827a6aba9506b74d"
format_stamp: "Formatted at 2026-08-11 05:16:08 on dist-test-slave-04bb"
server_key: "2810674ecc90ffeff2f86aee2926efdb"
server_key_iv: "c36eeb40584d0ae6356d46c109de9f1b"
server_key_version: "encryptionkey@0"
I20260811 05:16:09.541061 31059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:09.569736 31059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:09.572980 31059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:09.573192 31059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:09.573836 31059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:09.574728 31059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:16:09.574796 31059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:09.574868 31059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:16:09.574911 31059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:09.584196 31059 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:35611
I20260811 05:16:09.584223 31267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:35611 every 8 connection(s)
I20260811 05:16:09.585309 31059 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:16:09.591213 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 31059
Aug 11 05:16:09 dist-test-slave-04bb krb5kdc[29505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425369, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:16:09.600181 31279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:09.592361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33144 (local address 127.7.67.190:38739)
0811 05:16:09.592719 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:09.592725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:09.592739 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:16:09.592855 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:09.592858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:09.592907 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:09.593002 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:09.593796 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.594297 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.595154 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.595383 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.595486 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:09.596922 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:09.596926 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:09.596928 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:09.596972 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:09.598391 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:09.598806 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:09.598810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:09.598812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:09.598896 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:09.599171 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:09.599174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:09.599176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:09.599378 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:09.599589 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:09.599809 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 05:16:09.599952 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":277,"spinlock_wait_cycles":768,"thread_start_us":105,"threads_started":1}
I20260811 05:16:09.600347 30810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:09.587210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:39145 (local address 127.7.67.190:38739)
0811 05:16:09.587389 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:09.587393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:09.588247 (+ 854us) server_negotiation.cc:407] Connection header received
0811 05:16:09.589672 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:09.589676 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:09.589737 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:09.589831 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:09.591106 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.592026 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.593601 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.593972 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.594090 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:09.596618 (+ 2528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:09.596621 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:09.596624 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:09.596655 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:09.598252 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:09.598762 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:09.598766 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:09.598768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:09.598852 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:09.599187 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:09.599191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:09.599192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:09.599384 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:09.599487 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:09.600037 (+ 550us) server_negotiation.cc:299] Negotiation successful
0811 05:16:09.600145 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":85}
I20260811 05:16:09.601332 31274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:09.587616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.131:39145)
0811 05:16:09.588095 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:09.588137 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:09.589451 (+ 1314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:09.590008 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:09.590015 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:09.590392 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:09.590931 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:09.590951 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.592213 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.592218 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:09.593474 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:09.593485 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.594132 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.594137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:09.594322 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:09.594942 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:09.594959 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:09.596413 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:09.598369 (+ 1956us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:09.598375 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:09.598391 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:09.598648 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:09.598942 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:09.598945 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:09.598947 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:09.599087 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:09.599487 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:09.599495 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:09.599786 (+ 291us) client_negotiation.cc:769] Sending connection context
0811 05:16:09.599966 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 05:16:09.600308 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":131,"threads_started":1}
I20260811 05:16:09.602442 31268 heartbeater.cc:347] Connected to a master server at 127.7.67.190:38739
I20260811 05:16:09.602685 31268 heartbeater.cc:464] Registering TS with master...
I20260811 05:16:09.603178 31268 heartbeater.cc:511] Master 127.7.67.190:38739 requested a full tablet report, sending...
I20260811 05:16:09.604158 30480 ts_manager.cc:194] Registered new tserver with Master: d4db37d43e274283827a6aba9506b74d (127.7.67.131:35611)
I20260811 05:16:09.604815 30480 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.131@KRBTEST.COM'} at 127.7.67.131:39145
I20260811 05:16:09.614296 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:16:09.618973 31279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:09.614947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33148 (local address 127.7.67.190:38739)
0811 05:16:09.615068 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:09.615072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:09.615308 (+ 236us) server_negotiation.cc:407] Connection header received
0811 05:16:09.615462 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:09.615465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:09.615570 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:09.615673 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:16:09.615892 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.616384 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.617441 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.617665 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.617745 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:09.618106 (+ 361us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:16:09.618568 (+ 462us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:16:09.618659 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:09.618812 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 05:16:09.618866 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 05:16:09.619803 30480 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:33148:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:16:09.623135 30480 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:16:09.631326 30687 tablet_service.cc:1467] Processing CreateTablet for tablet 0dd05518b90a497bb1e7a1f404722d51 (DEFAULT_TABLE table=test-table [id=72667673f1d2498aaa8a8b95b00b1cf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:16:09.632050 30687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dd05518b90a497bb1e7a1f404722d51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:09.635052 30618 tablet_bootstrap.cc:492] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b: Bootstrap starting.
I20260811 05:16:09.635991 30618 tablet_bootstrap.cc:654] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:09.636287 30817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:09.630162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:35611 (local address 127.0.0.1:57462)
0811 05:16:09.630385 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:09.630402 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:09.630494 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:09.631495 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:09.631498 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:09.631522 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:09.631624 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:09.631630 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.633192 (+ 1562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.633195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:09.634407 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:09.634418 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.635765 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.635769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:09.635873 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:09.635918 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 05:16:09.636018 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 05:16:09.636121 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109}
I20260811 05:16:09.636572 31286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:09.630595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:44347 (local address 127.0.0.1:54962)
0811 05:16:09.630990 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:09.631004 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:09.631094 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:09.631982 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:09.631986 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:09.632005 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:09.632102 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:09.632108 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.633786 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.633790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:09.634699 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:09.634711 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.636080 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.636087 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:09.636186 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:09.636210 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 05:16:09.636272 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 05:16:09.636377 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":2,"thread_start_us":84,"threads_started":1}
I20260811 05:16:09.636739 31285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:09.630327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57462 (local address 127.7.67.131:35611)
0811 05:16:09.630852 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:09.630869 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:09.630900 (+ 31us) server_negotiation.cc:407] Connection header received
0811 05:16:09.630991 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:09.631000 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:09.631317 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:09.631501 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:09.631762 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.633022 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.634550 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.635599 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.635767 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:09.635938 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:09.636183 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 05:16:09.636286 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":188,"threads_started":1}
I20260811 05:16:09.637353 31289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:09.630998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54962 (local address 127.7.67.130:44347)
0811 05:16:09.631386 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:09.631401 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:09.631428 (+ 27us) server_negotiation.cc:407] Connection header received
0811 05:16:09.631509 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:09.631514 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:09.631825 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:09.632001 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:09.632551 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.633629 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.634850 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.635932 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.636866 (+ 934us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:09.637017 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:09.637068 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 05:16:09.637158 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":136,"threads_started":1}
I20260811 05:16:09.637874 30618 tablet_bootstrap.cc:492] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b: No bootstrap required, opened a new log
I20260811 05:16:09.637969 30618 ts_tablet_manager.cc:1397] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 05:16:09.638381 30618 raft_consensus.cc:348] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } }
I20260811 05:16:09.638500 30618 raft_consensus.cc:374] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:09.638548 30618 raft_consensus.cc:729] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdabab5544a445619a9e3ceee6af4e6b, State: Initialized, Role: FOLLOWER
I20260811 05:16:09.638677 30618 consensus_queue.cc:260] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } }
I20260811 05:16:09.638936 30618 ts_tablet_manager.cc:1428] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:16:09.638763 30938 tablet_service.cc:1467] Processing CreateTablet for tablet 0dd05518b90a497bb1e7a1f404722d51 (DEFAULT_TABLE table=test-table [id=72667673f1d2498aaa8a8b95b00b1cf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:16:09.639961 30938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dd05518b90a497bb1e7a1f404722d51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:09.640125 31165 tablet_service.cc:1467] Processing CreateTablet for tablet 0dd05518b90a497bb1e7a1f404722d51 (DEFAULT_TABLE table=test-table [id=72667673f1d2498aaa8a8b95b00b1cf3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:16:09.641311 31165 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0dd05518b90a497bb1e7a1f404722d51. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:09.645804 30865 tablet_bootstrap.cc:492] T 0dd05518b90a497bb1e7a1f404722d51 P b11eb4abed604862a99c16f06ff8f139: Bootstrap starting.
I20260811 05:16:09.647733 30865 tablet_bootstrap.cc:654] T 0dd05518b90a497bb1e7a1f404722d51 P b11eb4abed604862a99c16f06ff8f139: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:09.648559 30865 log.cc:824] T 0dd05518b90a497bb1e7a1f404722d51 P b11eb4abed604862a99c16f06ff8f139: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:09.650022 31292 tablet_bootstrap.cc:492] T 0dd05518b90a497bb1e7a1f404722d51 P d4db37d43e274283827a6aba9506b74d: Bootstrap starting.
I20260811 05:16:09.650637 30865 tablet_bootstrap.cc:492] T 0dd05518b90a497bb1e7a1f404722d51 P b11eb4abed604862a99c16f06ff8f139: No bootstrap required, opened a new log
I20260811 05:16:09.650833 30865 ts_tablet_manager.cc:1397] T 0dd05518b90a497bb1e7a1f404722d51 P b11eb4abed604862a99c16f06ff8f139: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 05:16:09.652240 31292 tablet_bootstrap.cc:654] T 0dd05518b90a497bb1e7a1f404722d51 P d4db37d43e274283827a6aba9506b74d: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:09.653549 31292 log.cc:824] T 0dd05518b90a497bb1e7a1f404722d51 P d4db37d43e274283827a6aba9506b74d: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:09.654844 30865 raft_consensus.cc:348] T 0dd05518b90a497bb1e7a1f404722d51 P b11eb4abed604862a99c16f06ff8f139 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } }
I20260811 05:16:09.655128 30865 raft_consensus.cc:374] T 0dd05518b90a497bb1e7a1f404722d51 P b11eb4abed604862a99c16f06ff8f139 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:09.655192 30865 raft_consensus.cc:729] T 0dd05518b90a497bb1e7a1f404722d51 P b11eb4abed604862a99c16f06ff8f139 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b11eb4abed604862a99c16f06ff8f139, State: Initialized, Role: FOLLOWER
I20260811 05:16:09.655515 30816 raft_consensus.cc:482] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:16:09.655673 30865 consensus_queue.cc:260] T 0dd05518b90a497bb1e7a1f404722d51 P b11eb4abed604862a99c16f06ff8f139 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } }
I20260811 05:16:09.655691 30816 raft_consensus.cc:504] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } }
I20260811 05:16:09.656323 31055 heartbeater.cc:503] Master 127.7.67.190:38739 was elected leader, sending a full tablet report...
I20260811 05:16:09.656730 30865 ts_tablet_manager.cc:1428] T 0dd05518b90a497bb1e7a1f404722d51 P b11eb4abed604862a99c16f06ff8f139: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 05:16:09.657150 31292 tablet_bootstrap.cc:492] T 0dd05518b90a497bb1e7a1f404722d51 P d4db37d43e274283827a6aba9506b74d: No bootstrap required, opened a new log
I20260811 05:16:09.657425 31292 ts_tablet_manager.cc:1397] T 0dd05518b90a497bb1e7a1f404722d51 P d4db37d43e274283827a6aba9506b74d: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
W20260811 05:16:09.657907 30808 tablet.cc:2369] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:16:09.656991 30816 leader_election.cc:288] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4db37d43e274283827a6aba9506b74d (127.7.67.131:35611), b11eb4abed604862a99c16f06ff8f139 (127.7.67.130:44347)
I20260811 05:16:09.661562 31292 raft_consensus.cc:348] T 0dd05518b90a497bb1e7a1f404722d51 P d4db37d43e274283827a6aba9506b74d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } }
I20260811 05:16:09.661857 31292 raft_consensus.cc:374] T 0dd05518b90a497bb1e7a1f404722d51 P d4db37d43e274283827a6aba9506b74d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:09.661914 31292 raft_consensus.cc:729] T 0dd05518b90a497bb1e7a1f404722d51 P d4db37d43e274283827a6aba9506b74d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4db37d43e274283827a6aba9506b74d, State: Initialized, Role: FOLLOWER
I20260811 05:16:09.662441 31289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:09.658443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:54475 (local address 127.7.67.130:44347)
0811 05:16:09.658548 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:09.658553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:09.658653 (+ 100us) server_negotiation.cc:407] Connection header received
0811 05:16:09.658816 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:09.658819 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:09.658864 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:09.658927 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:09.659311 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.659961 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.661480 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.661875 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.662006 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:09.662058 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:09.662264 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 05:16:09.662339 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 05:16:09.662361 31292 consensus_queue.cc:260] T 0dd05518b90a497bb1e7a1f404722d51 P d4db37d43e274283827a6aba9506b74d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } }
I20260811 05:16:09.662477 31297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:09.658111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:44347 (local address 127.7.67.129:54475)
0811 05:16:09.658571 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:09.658584 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:09.658691 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:09.659046 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:09.659049 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:09.659068 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:09.659180 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:09.659187 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.660093 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.660098 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:09.661352 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:09.661362 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.662008 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.662012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:09.662103 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:09.662160 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 05:16:09.662237 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 05:16:09.662311 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":397,"mutex_wait_us":63,"thread_start_us":74,"threads_started":1}
I20260811 05:16:09.663067 31268 heartbeater.cc:503] Master 127.7.67.190:38739 was elected leader, sending a full tablet report...
I20260811 05:16:09.663056 30965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dd05518b90a497bb1e7a1f404722d51" candidate_uuid: "bdabab5544a445619a9e3ceee6af4e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b11eb4abed604862a99c16f06ff8f139" is_pre_election: true
I20260811 05:16:09.663322 30965 raft_consensus.cc:2454] T 0dd05518b90a497bb1e7a1f404722d51 P b11eb4abed604862a99c16f06ff8f139 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdabab5544a445619a9e3ceee6af4e6b in term 0.
I20260811 05:16:09.663367 30811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:09.657144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:35611 (local address 127.7.67.129:49365)
0811 05:16:09.657307 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:09.657323 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:09.657423 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:09.658827 (+ 1404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:09.658830 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:09.658851 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:09.658973 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:09.658980 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.660142 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.660145 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:09.661450 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:09.661461 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.662975 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.662978 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:09.663085 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:09.663109 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 05:16:09.663183 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 05:16:09.663250 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":61}
I20260811 05:16:09.663527 31292 ts_tablet_manager.cc:1428] T 0dd05518b90a497bb1e7a1f404722d51 P d4db37d43e274283827a6aba9506b74d: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:16:09.663770 30609 leader_election.cc:302] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b11eb4abed604862a99c16f06ff8f139, bdabab5544a445619a9e3ceee6af4e6b; no voters:
I20260811 05:16:09.664153 31285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:09.657626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:49365 (local address 127.7.67.131:35611)
0811 05:16:09.657769 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:09.657774 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:09.657790 (+ 16us) server_negotiation.cc:407] Connection header received
0811 05:16:09.657836 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:09.657839 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:09.657884 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:09.657979 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:09.659230 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.660001 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.661792 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.662381 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.662498 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:09.662550 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:09.663957 (+ 1407us) server_negotiation.cc:299] Negotiation successful
0811 05:16:09.664067 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 05:16:09.664232 30816 raft_consensus.cc:2781] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:16:09.664332 30816 raft_consensus.cc:482] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:16:09.664389 30816 raft_consensus.cc:3037] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:09.665560 30816 raft_consensus.cc:504] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } }
I20260811 05:16:09.665926 30816 leader_election.cc:288] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [CANDIDATE]: Term 1 election: Requested vote from peers d4db37d43e274283827a6aba9506b74d (127.7.67.131:35611), b11eb4abed604862a99c16f06ff8f139 (127.7.67.130:44347)
I20260811 05:16:09.666517 30965 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dd05518b90a497bb1e7a1f404722d51" candidate_uuid: "bdabab5544a445619a9e3ceee6af4e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b11eb4abed604862a99c16f06ff8f139"
I20260811 05:16:09.666546 31193 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dd05518b90a497bb1e7a1f404722d51" candidate_uuid: "bdabab5544a445619a9e3ceee6af4e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4db37d43e274283827a6aba9506b74d" is_pre_election: true
I20260811 05:16:09.666554 31191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0dd05518b90a497bb1e7a1f404722d51" candidate_uuid: "bdabab5544a445619a9e3ceee6af4e6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4db37d43e274283827a6aba9506b74d"
I20260811 05:16:09.666636 30965 raft_consensus.cc:3037] T 0dd05518b90a497bb1e7a1f404722d51 P b11eb4abed604862a99c16f06ff8f139 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:09.666796 31193 raft_consensus.cc:2454] T 0dd05518b90a497bb1e7a1f404722d51 P d4db37d43e274283827a6aba9506b74d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bdabab5544a445619a9e3ceee6af4e6b in term 0.
I20260811 05:16:09.667634 30965 raft_consensus.cc:2454] T 0dd05518b90a497bb1e7a1f404722d51 P b11eb4abed604862a99c16f06ff8f139 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bdabab5544a445619a9e3ceee6af4e6b in term 1.
I20260811 05:16:09.667927 30609 leader_election.cc:302] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b11eb4abed604862a99c16f06ff8f139, bdabab5544a445619a9e3ceee6af4e6b; no voters:
I20260811 05:16:09.668092 30816 raft_consensus.cc:2781] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:16:09.668313 30816 raft_consensus.cc:686] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 1 LEADER]: Becoming Leader. State: Replica: bdabab5544a445619a9e3ceee6af4e6b, State: Running, Role: LEADER
I20260811 05:16:09.668609 30816 consensus_queue.cc:237] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } }
I20260811 05:16:09.671478 30480 catalog_manager.cc:5495] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b reported cstate change: term changed from 0 to 1, leader changed from <none> to bdabab5544a445619a9e3ceee6af4e6b (127.7.67.129). New cstate: current_term: 1 leader_uuid: "bdabab5544a445619a9e3ceee6af4e6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } health_report { overall_health: HEALTHY } } }
I20260811 05:16:09.686580 30823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:09.682987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46008 (local address 127.7.67.129:42341)
0811 05:16:09.683106 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:09.683110 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:09.683124 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:16:09.683324 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:09.683327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:09.683383 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:09.683477 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:16:09.683720 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.684271 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.685198 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:09.685438 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:09.685521 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:09.685796 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:16:09.686223 (+ 427us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:16:09.686321 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:09.686418 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 05:16:09.686471 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 05:16:09.690555 31191 raft_consensus.cc:3037] T 0dd05518b90a497bb1e7a1f404722d51 P d4db37d43e274283827a6aba9506b74d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:09.690719 30965 raft_consensus.cc:1261] T 0dd05518b90a497bb1e7a1f404722d51 P b11eb4abed604862a99c16f06ff8f139 [term 1 FOLLOWER]: Refusing update from remote peer bdabab5544a445619a9e3ceee6af4e6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:16:09.691454 31301 consensus_queue.cc:1035] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:16:09.692050 31191 raft_consensus.cc:1261] T 0dd05518b90a497bb1e7a1f404722d51 P d4db37d43e274283827a6aba9506b74d [term 1 FOLLOWER]: Refusing update from remote peer bdabab5544a445619a9e3ceee6af4e6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:16:09.692696 31301 consensus_queue.cc:1035] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:16:09.708495 30481 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:33148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:16:09.708724 30481 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:33148:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:16:09.711253 30481 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e: Sending DeleteTablet for 3 replicas of tablet 0dd05518b90a497bb1e7a1f404722d51
I20260811 05:16:09.711941 30687 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dd05518b90a497bb1e7a1f404722d51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:16:09 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:45996
I20260811 05:16:09.711961 31165 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dd05518b90a497bb1e7a1f404722d51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:16:09 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:57462
I20260811 05:16:09.712029 30938 tablet_service.cc:1514] Processing DeleteTablet for tablet 0dd05518b90a497bb1e7a1f404722d51 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:16:09 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:54962
I20260811 05:16:09.712502 31321 tablet_replica.cc:330] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b: stopping tablet replica
I20260811 05:16:09.712678 31321 raft_consensus.cc:2229] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:16:09.712808 31324 tablet_replica.cc:330] T 0dd05518b90a497bb1e7a1f404722d51 P b11eb4abed604862a99c16f06ff8f139: stopping tablet replica
I20260811 05:16:09.712826 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 30546
I20260811 05:16:09.712980 31321 raft_consensus.cc:2258] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:16:09.713135 31324 raft_consensus.cc:2229] T 0dd05518b90a497bb1e7a1f404722d51 P b11eb4abed604862a99c16f06ff8f139 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:16:09.713433 31324 raft_consensus.cc:2258] T 0dd05518b90a497bb1e7a1f404722d51 P b11eb4abed604862a99c16f06ff8f139 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:16:09.720436 31324 ts_tablet_manager.cc:1905] T 0dd05518b90a497bb1e7a1f404722d51 P b11eb4abed604862a99c16f06ff8f139: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:16:09.722436 31322 tablet_replica.cc:330] T 0dd05518b90a497bb1e7a1f404722d51 P d4db37d43e274283827a6aba9506b74d: stopping tablet replica
I20260811 05:16:09.722776 31322 raft_consensus.cc:2229] T 0dd05518b90a497bb1e7a1f404722d51 P d4db37d43e274283827a6aba9506b74d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:16:09.723142 31322 raft_consensus.cc:2258] T 0dd05518b90a497bb1e7a1f404722d51 P d4db37d43e274283827a6aba9506b74d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:16:09.723424 31324 ts_tablet_manager.cc:1918] T 0dd05518b90a497bb1e7a1f404722d51 P b11eb4abed604862a99c16f06ff8f139: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:16:09.723500 31324 log.cc:1196] T 0dd05518b90a497bb1e7a1f404722d51 P b11eb4abed604862a99c16f06ff8f139: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/0dd05518b90a497bb1e7a1f404722d51
I20260811 05:16:09.723763 31324 ts_tablet_manager.cc:1939] T 0dd05518b90a497bb1e7a1f404722d51 P b11eb4abed604862a99c16f06ff8f139: Deleting consensus metadata
I20260811 05:16:09.725143 31322 ts_tablet_manager.cc:1905] T 0dd05518b90a497bb1e7a1f404722d51 P d4db37d43e274283827a6aba9506b74d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:16:09.726359 30458 catalog_manager.cc:4841] TS b11eb4abed604862a99c16f06ff8f139 (127.7.67.130:44347): tablet 0dd05518b90a497bb1e7a1f404722d51 (table test-table [id=72667673f1d2498aaa8a8b95b00b1cf3]) successfully deleted
I20260811 05:16:09.727319 31322 ts_tablet_manager.cc:1918] T 0dd05518b90a497bb1e7a1f404722d51 P d4db37d43e274283827a6aba9506b74d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:16:09.727415 31322 log.cc:1196] T 0dd05518b90a497bb1e7a1f404722d51 P d4db37d43e274283827a6aba9506b74d: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/0dd05518b90a497bb1e7a1f404722d51
I20260811 05:16:09.727696 31322 ts_tablet_manager.cc:1939] T 0dd05518b90a497bb1e7a1f404722d51 P d4db37d43e274283827a6aba9506b74d: Deleting consensus metadata
I20260811 05:16:09.728826 30459 catalog_manager.cc:4841] TS d4db37d43e274283827a6aba9506b74d (127.7.67.131:35611): tablet 0dd05518b90a497bb1e7a1f404722d51 (table test-table [id=72667673f1d2498aaa8a8b95b00b1cf3]) successfully deleted
W20260811 05:16:09.734421 30460 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.129:42341 (error 108)
I20260811 05:16:09.735294 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 30814
I20260811 05:16:09.735544 31286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:09.735218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:42341 (local address 127.0.0.1:46010)
0811 05:16:09.735352 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:09.735443 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.129:42341: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 05:16:09.739238 30460 catalog_manager.cc:4584] TS bdabab5544a445619a9e3ceee6af4e6b (127.7.67.129:42341): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0dd05518b90a497bb1e7a1f404722d51: Network error: Client connection negotiation failed: client connection to 127.7.67.129:42341: connect: Connection refused (error 111)
I20260811 05:16:09.745541 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 31059
I20260811 05:16:09.760171 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 30424
I20260811 05:16:09.775821 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:38739
--webserver_interface=127.7.67.190
--webserver_port=33627
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:38195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:38739
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:16:09.896447 31326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:09.896795 31326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:09.896860 31326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:09.900866 31326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:16:09.900916 31326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:09.900935 31326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:09.900952 31326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:16:09.901001 31326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:16:09.906368 31326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:38195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:38739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:38739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=33627
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:16:09.907433 31326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:09.908490 31326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:09.917229 31335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:09.917243 31333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:09.917244 31338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:09.917495 31326 server_base.cc:1034] running on GCE node
I20260811 05:16:09.918102 31326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:09.919227 31326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:09.920415 31326 hybrid_clock.cc:648] HybridClock initialized: now 1786425369920383 us; error 47 us; skew 500 ppm
Aug 11 05:16:09 dist-test-slave-04bb krb5kdc[29505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425369, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:09.923303 31326 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:16:09.924234 31326 webserver.cc:460] Webserver started at http://127.7.67.190:33627/ using document root <none> and password file <none>
I20260811 05:16:09.924865 31326 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:09.924954 31326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:09.929145 31326 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.002s
I20260811 05:16:09.931474 31349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:09.932873 31326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:16:09.933017 31326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "0c8a1f31a2f9487ab869de998b28f97e"
format_stamp: "Formatted at 2026-08-11 05:16:07 on dist-test-slave-04bb"
server_key: "b85f448a35f1f34828dc85dea019e4fb"
server_key_iv: "cda249c7ad42af3d947ec61529234837"
server_key_version: "encryptionkey@0"
I20260811 05:16:09.933585 31326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:09.962006 31326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:09.965618 31326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:09.965842 31326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:09.973392 31326 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:38739
I20260811 05:16:09.973672 31439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:38739 every 8 connection(s)
I20260811 05:16:09.974541 31326 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:16:09.978999 31440 sys_catalog.cc:263] Verifying existing consensus state
I20260811 05:16:09.980754 31440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e: Bootstrap starting.
I20260811 05:16:09.983618 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 31326
I20260811 05:16:09.984102 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:42341
--local_ip_for_outbound_sockets=127.7.67.129
--tserver_master_addrs=127.7.67.190:38739
--webserver_port=42163
--webserver_interface=127.7.67.129
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:38195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:16:09.998790 31440 log.cc:824] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:10.010550 31440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:16:10.010938 31440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e: Bootstrap complete.
I20260811 05:16:10.015094 31440 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8a1f31a2f9487ab869de998b28f97e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38739 } }
I20260811 05:16:10.015517 31440 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c8a1f31a2f9487ab869de998b28f97e, State: Initialized, Role: FOLLOWER
I20260811 05:16:10.016052 31440 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8a1f31a2f9487ab869de998b28f97e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38739 } }
I20260811 05:16:10.016204 31440 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:16:10.016296 31440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:16:10.016414 31440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 2 FOLLOWER]: Advancing to term 3
I20260811 05:16:10.018352 31440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8a1f31a2f9487ab869de998b28f97e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38739 } }
I20260811 05:16:10.018695 31440 leader_election.cc:302] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c8a1f31a2f9487ab869de998b28f97e; no voters:
I20260811 05:16:10.019052 31440 leader_election.cc:288] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 05:16:10.019155 31450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 3 FOLLOWER]: Leader election won for term 3
I20260811 05:16:10.019405 31450 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [term 3 LEADER]: Becoming Leader. State: Replica: 0c8a1f31a2f9487ab869de998b28f97e, State: Running, Role: LEADER
I20260811 05:16:10.019706 31450 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8a1f31a2f9487ab869de998b28f97e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38739 } }
I20260811 05:16:10.020371 31440 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:16:10.020664 31450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c8a1f31a2f9487ab869de998b28f97e. Latest consensus state: current_term: 3 leader_uuid: "0c8a1f31a2f9487ab869de998b28f97e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8a1f31a2f9487ab869de998b28f97e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38739 } } }
I20260811 05:16:10.020800 31450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [sys.catalog]: This master's current role is: LEADER
I20260811 05:16:10.021206 31451 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0c8a1f31a2f9487ab869de998b28f97e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c8a1f31a2f9487ab869de998b28f97e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38739 } } }
I20260811 05:16:10.021385 31451 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c8a1f31a2f9487ab869de998b28f97e [sys.catalog]: This master's current role is: LEADER
I20260811 05:16:10.022487 31457 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:16:10.027781 31457 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:16:10.028465 31457 catalog_manager.cc:1259] Loaded cluster ID: 49ebe35220b74d2f905142fc77f917f8
I20260811 05:16:10.028546 31457 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 05:16:10.029763 31457 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f7badf559ad google::LogMessage::Fail() at ??:0
@ 0x7f7badf5888a google::LogMessage::SendToLog() at ??:0
@ 0x7f7badf554de google::LogMessage::Flush() at ??:0
@ 0x7f7badf56a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f7bb8894fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f7bb8895d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f7bb8892579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f7bb88d6640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f7bb01a73d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f7bb01a7c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f7bb01a93f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f7bb0198512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f7bb17dc6db start_thread at ??:0
@ 0x7f7baca5361f clone at ??:0
W20260811 05:16:10.146767 31448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:10.147030 31448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:10.147076 31448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:10.151165 31448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:10.151221 31448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:10.151314 31448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:16:10.156710 31448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:38195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:42341
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=42163
--tserver_master_addrs=127.7.67.190:38739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:16:10.157794 31448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:10.158927 31448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:10.169207 31520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:10.169231 31522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:10.169216 31518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:10.169648 31448 server_base.cc:1034] running on GCE node
I20260811 05:16:10.170042 31448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:10.170706 31448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:10.171887 31448 hybrid_clock.cc:648] HybridClock initialized: now 1786425370171857 us; error 50 us; skew 500 ppm
Aug 11 05:16:10 dist-test-slave-04bb krb5kdc[29505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425370, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:10.175009 31448 init.cc:377] Logged in from keytab as kudu/127.7.67.129@KRBTEST.COM (short username kudu)
I20260811 05:16:10.175985 31448 webserver.cc:460] Webserver started at http://127.7.67.129:42163/ using document root <none> and password file <none>
I20260811 05:16:10.176645 31448 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:10.176714 31448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:10.180799 31448 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:16:10.183111 31533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:10.184422 31448 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:16:10.184561 31448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "bdabab5544a445619a9e3ceee6af4e6b"
format_stamp: "Formatted at 2026-08-11 05:16:07 on dist-test-slave-04bb"
server_key: "342143cf6cd5e69411c1f1b3e3297ea0"
server_key_iv: "f9dcdc70efd59352e1170b5035a9a2fc"
server_key_version: "encryptionkey@0"
I20260811 05:16:10.185093 31448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:10.194624 31448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:10.197861 31448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:10.198055 31448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:10.198691 31448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:10.199815 31541 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 05:16:10.204298 31448 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 05:16:10.204365 31448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260811 05:16:10.204433 31448 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 05:16:10.206175 31448 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 05:16:10.206233 31448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 05:16:10.206511 31541 tablet_bootstrap.cc:492] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b: Bootstrap starting.
I20260811 05:16:10.217555 31448 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:42341
I20260811 05:16:10.217782 31727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:42341 every 8 connection(s)
I20260811 05:16:10.218824 31448 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:16:10.222607 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 31448
I20260811 05:16:10.223142 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:44347
--local_ip_for_outbound_sockets=127.7.67.130
--tserver_master_addrs=127.7.67.190:38739
--webserver_port=43285
--webserver_interface=127.7.67.130
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:38195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:16:10.228289 31541 log.cc:824] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:10.238726 31541 tablet_bootstrap.cc:492] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:16:10.239161 31541 tablet_bootstrap.cc:492] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b: Bootstrap complete.
I20260811 05:16:10.240655 31541 ts_tablet_manager.cc:1397] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b: Time spent bootstrapping tablet: real 0.034s user 0.022s sys 0.012s
I20260811 05:16:10.243979 31541 raft_consensus.cc:348] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } }
I20260811 05:16:10.244577 31541 raft_consensus.cc:729] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bdabab5544a445619a9e3ceee6af4e6b, State: Initialized, Role: FOLLOWER
I20260811 05:16:10.245186 31541 consensus_queue.cc:260] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } }
I20260811 05:16:10.246337 31541 ts_tablet_manager.cc:1428] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b: Time spent starting tablet: real 0.006s user 0.001s sys 0.004s
W20260811 05:16:10.263872 31731 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:16:10.221268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.129:57823)
0811 05:16:10.221918 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:10.221980 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:10.223134 (+ 1154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:10.262977 (+ 39843us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":183,"threads_started":1}
W20260811 05:16:10.264485 31729 heartbeater.cc:644] Failed to heartbeat to 127.7.67.190:38739 (0 consecutive failures): Network error: Failed to ping master at 127.7.67.190:38739: Client connection negotiation failed: client connection to 127.7.67.190:38739: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 05:16:10.265326 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:10.265051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.129:35085)
0811 05:16:10.265190 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:10.265263 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 05:16:10.266049 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:10.265836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.129:50139)
0811 05:16:10.265949 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:10.265999 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 05:16:10.266285 31729 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 05:16:10.348902 31732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:10.349218 31732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:10.349311 31732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:10.353395 31732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:10.353475 31732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:10.353566 31732 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:16:10.358949 31732 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:38195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:44347
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=43285
--tserver_master_addrs=127.7.67.190:38739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:16:10.360059 31732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:10.361147 31732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:10.371603 31753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:10.371600 31747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:10.371559 31748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:10.371814 31732 server_base.cc:1034] running on GCE node
I20260811 05:16:10.372498 31732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:10.373250 31732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:10.374459 31732 hybrid_clock.cc:648] HybridClock initialized: now 1786425370374436 us; error 33 us; skew 500 ppm
Aug 11 05:16:10 dist-test-slave-04bb krb5kdc[29505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425370, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:10.377530 31732 init.cc:377] Logged in from keytab as kudu/127.7.67.130@KRBTEST.COM (short username kudu)
I20260811 05:16:10.378495 31732 webserver.cc:460] Webserver started at http://127.7.67.130:43285/ using document root <none> and password file <none>
I20260811 05:16:10.379091 31732 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:10.379230 31732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:10.383347 31732 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 05:16:10.385622 31763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:10.386929 31732 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:16:10.387084 31732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "b11eb4abed604862a99c16f06ff8f139"
format_stamp: "Formatted at 2026-08-11 05:16:08 on dist-test-slave-04bb"
server_key: "8622d41f7f1143006d7c046d6800b6a2"
server_key_iv: "91fd5ae090230deee00bb0d30dadf26c"
server_key_version: "encryptionkey@0"
I20260811 05:16:10.387678 31732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:10.410642 31732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:10.414078 31732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:10.414247 31732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:10.414822 31732 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:10.415663 31732 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:16:10.415720 31732 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:10.415762 31732 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:16:10.415777 31732 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:10.424923 31732 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:44347
I20260811 05:16:10.425019 31959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:44347 every 8 connection(s)
I20260811 05:16:10.425997 31732 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:16:10.430148 31968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:10.428221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.130:42785)
0811 05:16:10.428733 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:10.428845 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":112,"threads_started":1}
W20260811 05:16:10.430600 31961 heartbeater.cc:644] Failed to heartbeat to 127.7.67.190:38739 (0 consecutive failures): Network error: Failed to ping master at 127.7.67.190:38739: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
I20260811 05:16:10.431221 31968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:10.430991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.130:47067)
0811 05:16:10.431105 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:10.431161 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 05:16:10.431829 31968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:10.431619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.130:42255)
0811 05:16:10.431722 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:10.431771 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 05:16:10.431998 31961 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 05:16:10.432994 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 31732
I20260811 05:16:10.433456 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:35611
--local_ip_for_outbound_sockets=127.7.67.131
--tserver_master_addrs=127.7.67.190:38739
--webserver_port=41533
--webserver_interface=127.7.67.131
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:38195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:16:10.550171 31976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:10.550444 31976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:10.550488 31976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:10.554589 31976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:10.554652 31976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:10.554710 31976 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:16:10.560192 31976 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:38195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:35611
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=41533
--tserver_master_addrs=127.7.67.190:38739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:16:10.561335 31976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:10.562438 31976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:10.572950 31989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:10.573081 31976 server_base.cc:1034] running on GCE node
W20260811 05:16:10.572964 31985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:10.572964 31987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:10.573644 31976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:10.574318 31976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:10.575527 31976 hybrid_clock.cc:648] HybridClock initialized: now 1786425370575504 us; error 32 us; skew 500 ppm
Aug 11 05:16:10 dist-test-slave-04bb krb5kdc[29505](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425370, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:10.578665 31976 init.cc:377] Logged in from keytab as kudu/127.7.67.131@KRBTEST.COM (short username kudu)
I20260811 05:16:10.579638 31976 webserver.cc:460] Webserver started at http://127.7.67.131:41533/ using document root <none> and password file <none>
I20260811 05:16:10.580307 31976 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:10.580425 31976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:10.584503 31976 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:16:10.586799 32001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:10.588127 31976 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 05:16:10.588299 31976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "d4db37d43e274283827a6aba9506b74d"
format_stamp: "Formatted at 2026-08-11 05:16:08 on dist-test-slave-04bb"
server_key: "2810674ecc90ffeff2f86aee2926efdb"
server_key_iv: "c36eeb40584d0ae6356d46c109de9f1b"
server_key_version: "encryptionkey@0"
I20260811 05:16:10.588994 31976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:10.604879 31976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:10.608063 31976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:10.608306 31976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:10.608973 31976 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:10.609860 31976 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:16:10.609931 31976 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:10.610003 31976 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:16:10.610050 31976 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:10.619143 31976 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:35611
I20260811 05:16:10.619166 32192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:35611 every 8 connection(s)
I20260811 05:16:10.620164 31976 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:16:10.621152 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 31976
I20260811 05:16:10.624215 32197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:10.622470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.131:51887)
0811 05:16:10.622927 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:10.623031 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":119,"threads_started":1}
W20260811 05:16:10.624797 32194 heartbeater.cc:644] Failed to heartbeat to 127.7.67.190:38739 (0 consecutive failures): Network error: Failed to ping master at 127.7.67.190:38739: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
I20260811 05:16:10.625399 32197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:10.625169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.131:37961)
0811 05:16:10.625278 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:10.625332 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 05:16:10.626000 32197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:10.625810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.131:54557)
0811 05:16:10.625907 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:10.625948 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 05:16:10.626179 32194 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 05:16:11.267395 32198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:11.266910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.129:44329)
0811 05:16:11.267188 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:11.267250 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":83,"threads_started":1}
I20260811 05:16:11.433105 32199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:11.432626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.130:36201)
0811 05:16:11.432932 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:11.432992 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":78,"threads_started":1}
I20260811 05:16:11.627178 32200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:11.626732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.131:58123)
0811 05:16:11.627015 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:11.627075 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":74,"threads_started":1}
I20260811 05:16:12.120201 32202 raft_consensus.cc:482] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:16:12.120460 32202 raft_consensus.cc:504] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } }
I20260811 05:16:12.121716 32202 leader_election.cc:288] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4db37d43e274283827a6aba9506b74d (127.7.67.131:35611), b11eb4abed604862a99c16f06ff8f139 (127.7.67.130:44347)
Aug 11 05:16:12 dist-test-slave-04bb krb5kdc[29505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425370, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.131@KRBTEST.COM
Aug 11 05:16:12 dist-test-slave-04bb krb5kdc[29505](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425370, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.130@KRBTEST.COM
I20260811 05:16:12.146624 32203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:12.122027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:44347 (local address 127.7.67.129:56805)
0811 05:16:12.122374 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:12.122389 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:12.122512 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:12.124628 (+ 2116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:12.124636 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:12.125336 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:12.126316 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:12.126342 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:12.128343 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:12.128347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:12.130126 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:12.130140 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:12.131174 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:12.131179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:12.131533 (+ 354us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:12.138031 (+ 6498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:12.138065 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:12.140749 (+ 2684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:12.143266 (+ 2517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:12.143273 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:12.143290 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:12.143776 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:12.144281 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:12.144284 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:12.144286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:12.144562 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:12.145532 (+ 970us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:12.145544 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:12.145795 (+ 251us) client_negotiation.cc:769] Sending connection context
0811 05:16:12.145982 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 05:16:12.146439 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":576,"spinlock_wait_cycles":90240,"thread_start_us":67,"threads_started":1}
I20260811 05:16:12.146657 32205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:12.122121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:41377 (local address 127.7.67.131:35611)
0811 05:16:12.122616 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:12.122636 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:12.122680 (+ 44us) server_negotiation.cc:407] Connection header received
0811 05:16:12.123619 (+ 939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:12.123628 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:12.124098 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:12.124446 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:12.126523 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:12.128144 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:12.130281 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:12.131020 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:12.131196 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:12.140670 (+ 9474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:12.140677 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:12.140692 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:12.140727 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:12.143110 (+ 2383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:12.143937 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:12.143945 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:12.143952 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:12.144145 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:12.144699 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:12.144702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:12.144704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:12.145310 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:12.145467 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:12.146138 (+ 671us) server_negotiation.cc:299] Negotiation successful
0811 05:16:12.146450 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":136,"threads_started":1}
I20260811 05:16:12.146759 32207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:12.122275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:56805 (local address 127.7.67.130:44347)
0811 05:16:12.122615 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:12.122634 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:12.122680 (+ 46us) server_negotiation.cc:407] Connection header received
0811 05:16:12.123619 (+ 939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:12.123628 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:12.124098 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:12.124446 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:12.126523 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:12.128144 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:12.130281 (+ 2137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:12.131020 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:12.131196 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:12.140985 (+ 9789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:12.140996 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:12.141015 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:12.141060 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:12.143109 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:12.143937 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:12.143945 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:12.143952 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:12.144145 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:12.144697 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:12.144701 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:12.144703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:12.145310 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:12.145467 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:12.146326 (+ 859us) server_negotiation.cc:299] Negotiation successful
0811 05:16:12.146581 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":69,"threads_started":1}
I20260811 05:16:12.146623 32204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:12.121924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:35611 (local address 127.7.67.129:41377)
0811 05:16:12.122305 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:12.122324 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:12.122456 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:12.124632 (+ 2176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:12.124636 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:12.125335 (+ 699us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:12.126316 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:12.126342 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:12.128343 (+ 2001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:12.128347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:12.130126 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:12.130140 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:12.131206 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:12.131209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:12.131533 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:12.138031 (+ 6498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:12.138060 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:12.140324 (+ 2264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:12.143266 (+ 2942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:12.143273 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:12.143290 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:12.143776 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:12.144281 (+ 505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:12.144284 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:12.144286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:12.144557 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:12.145545 (+ 988us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:12.145547 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:12.145796 (+ 249us) client_negotiation.cc:769] Sending connection context
0811 05:16:12.145984 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 05:16:12.146387 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":514,"spinlock_wait_cycles":33024,"thread_start_us":109,"threads_started":1}
W20260811 05:16:12.149585 31536 leader_election.cc:341] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d4db37d43e274283827a6aba9506b74d (127.7.67.131:35611): Not found: Tablet not found: 0dd05518b90a497bb1e7a1f404722d51
W20260811 05:16:12.149780 31535 leader_election.cc:341] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b11eb4abed604862a99c16f06ff8f139 (127.7.67.130:44347): Not found: Tablet not found: 0dd05518b90a497bb1e7a1f404722d51
I20260811 05:16:12.149871 31535 leader_election.cc:302] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdabab5544a445619a9e3ceee6af4e6b; no voters: b11eb4abed604862a99c16f06ff8f139, d4db37d43e274283827a6aba9506b74d
I20260811 05:16:12.150120 32202 raft_consensus.cc:2726] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:16:12.268606 32204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:12.268250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.129:51407)
0811 05:16:12.268402 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:12.268459 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 05:16:12.434402 32208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:12.433903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.130:58795)
0811 05:16:12.434191 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:12.434267 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":78,"threads_started":1}
I20260811 05:16:12.628885 32209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:12.628283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.131:50883)
0811 05:16:12.628665 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:12.628737 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":116,"threads_started":1}
I20260811 05:16:13.269894 32210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:13.269465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.129:38445)
0811 05:16:13.269728 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:13.269792 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":81,"threads_started":1}
I20260811 05:16:13.435765 32211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:13.435286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.130:43729)
0811 05:16:13.435586 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:13.435662 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":100,"threads_started":1}
I20260811 05:16:13.630128 32212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:13.629663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.131:46681)
0811 05:16:13.629967 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:13.630029 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":92,"threads_started":1}
I20260811 05:16:13.982556 32213 raft_consensus.cc:482] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:16:13.982690 32213 raft_consensus.cc:504] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } }
I20260811 05:16:13.983210 32213 leader_election.cc:288] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4db37d43e274283827a6aba9506b74d (127.7.67.131:35611), b11eb4abed604862a99c16f06ff8f139 (127.7.67.130:44347)
W20260811 05:16:13.984032 31536 leader_election.cc:341] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d4db37d43e274283827a6aba9506b74d (127.7.67.131:35611): Not found: Tablet not found: 0dd05518b90a497bb1e7a1f404722d51
W20260811 05:16:13.984122 31535 leader_election.cc:341] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b11eb4abed604862a99c16f06ff8f139 (127.7.67.130:44347): Not found: Tablet not found: 0dd05518b90a497bb1e7a1f404722d51
I20260811 05:16:13.984246 31535 leader_election.cc:302] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdabab5544a445619a9e3ceee6af4e6b; no voters: b11eb4abed604862a99c16f06ff8f139, d4db37d43e274283827a6aba9506b74d
I20260811 05:16:13.984472 32213 raft_consensus.cc:2726] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:16:14.271166 32215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:14.270693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.129:34905)
0811 05:16:14.271001 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:14.271059 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":105,"threads_started":1}
I20260811 05:16:14.437135 32216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:14.436640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.130:38199)
0811 05:16:14.436962 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:14.437019 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":113,"threads_started":1}
I20260811 05:16:14.631417 32218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:14.630956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.131:33961)
0811 05:16:14.631250 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:14.631311 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":95,"threads_started":1}
I20260811 05:16:15.272852 32219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:15.272305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.129:54467)
0811 05:16:15.272684 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:15.272746 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":78,"threads_started":1}
I20260811 05:16:15.438447 32220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:15.438029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.130:52977)
0811 05:16:15.438295 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:15.438351 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":78,"threads_started":1}
I20260811 05:16:15.632841 32222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:15.632323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.131:38225)
0811 05:16:15.632638 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:15.632694 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":126,"threads_started":1}
I20260811 05:16:16.274271 32224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:16.273758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.129:57171)
0811 05:16:16.274089 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:16.274148 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":112,"threads_started":1}
I20260811 05:16:16.439692 32225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:16.439267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.130:40375)
0811 05:16:16.439535 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:16.439591 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":85,"threads_started":1}
I20260811 05:16:16.634228 32226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:16.633683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.131:43927)
0811 05:16:16.634025 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:16.634093 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":114,"threads_started":1}
I20260811 05:16:17.275703 32227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:17.275241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.129:57483)
0811 05:16:17.275537 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:17.275594 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":105,"threads_started":1}
I20260811 05:16:17.304471 32229 raft_consensus.cc:482] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:16:17.304694 32229 raft_consensus.cc:504] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } }
I20260811 05:16:17.305208 32229 leader_election.cc:288] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4db37d43e274283827a6aba9506b74d (127.7.67.131:35611), b11eb4abed604862a99c16f06ff8f139 (127.7.67.130:44347)
W20260811 05:16:17.306025 31536 leader_election.cc:341] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d4db37d43e274283827a6aba9506b74d (127.7.67.131:35611): Not found: Tablet not found: 0dd05518b90a497bb1e7a1f404722d51
W20260811 05:16:17.306133 31535 leader_election.cc:341] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b11eb4abed604862a99c16f06ff8f139 (127.7.67.130:44347): Not found: Tablet not found: 0dd05518b90a497bb1e7a1f404722d51
I20260811 05:16:17.306249 31535 leader_election.cc:302] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdabab5544a445619a9e3ceee6af4e6b; no voters: b11eb4abed604862a99c16f06ff8f139, d4db37d43e274283827a6aba9506b74d
I20260811 05:16:17.306448 32229 raft_consensus.cc:2726] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:16:17.441206 32233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:17.440508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.130:59995)
0811 05:16:17.440926 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:17.441021 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":99,"threads_started":1}
I20260811 05:16:17.635761 32234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:17.635142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.131:53649)
0811 05:16:17.635577 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:17.635656 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":165,"threads_started":1}
I20260811 05:16:18.277068 32237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:18.276646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.129:40565)
0811 05:16:18.276904 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:18.276965 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":76,"threads_started":1}
I20260811 05:16:18.442723 32238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:18.442160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.130:46415)
0811 05:16:18.442522 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:18.442592 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":111,"threads_started":1}
I20260811 05:16:18.637140 32239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:18.636656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.131:37403)
0811 05:16:18.636969 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:18.637040 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":80,"threads_started":1}
I20260811 05:16:19.278617 32242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:19.278075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.129:47227)
0811 05:16:19.278438 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:19.278511 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":88,"threads_started":1}
I20260811 05:16:19.444115 32246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:19.443648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.130:49249)
0811 05:16:19.443949 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:19.444014 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":80,"threads_started":1}
I20260811 05:16:19.638474 32247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:19.638018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.131:34123)
0811 05:16:19.638314 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:19.638374 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":82,"threads_started":1}
I20260811 05:16:20.280042 32251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:20.279595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.129:44533)
0811 05:16:20.279877 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:20.279939 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":82,"threads_started":1}
I20260811 05:16:20.437536 32252 raft_consensus.cc:482] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:16:20.437719 32252 raft_consensus.cc:504] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4db37d43e274283827a6aba9506b74d" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 35611 } } peers { permanent_uuid: "b11eb4abed604862a99c16f06ff8f139" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44347 } } peers { permanent_uuid: "bdabab5544a445619a9e3ceee6af4e6b" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42341 } }
I20260811 05:16:20.438175 32252 leader_election.cc:288] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d4db37d43e274283827a6aba9506b74d (127.7.67.131:35611), b11eb4abed604862a99c16f06ff8f139 (127.7.67.130:44347)
W20260811 05:16:20.438982 31536 leader_election.cc:341] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d4db37d43e274283827a6aba9506b74d (127.7.67.131:35611): Not found: Tablet not found: 0dd05518b90a497bb1e7a1f404722d51
W20260811 05:16:20.439118 31535 leader_election.cc:341] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b11eb4abed604862a99c16f06ff8f139 (127.7.67.130:44347): Not found: Tablet not found: 0dd05518b90a497bb1e7a1f404722d51
I20260811 05:16:20.439216 31535 leader_election.cc:302] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bdabab5544a445619a9e3ceee6af4e6b; no voters: b11eb4abed604862a99c16f06ff8f139, d4db37d43e274283827a6aba9506b74d
I20260811 05:16:20.439409 32252 raft_consensus.cc:2726] T 0dd05518b90a497bb1e7a1f404722d51 P bdabab5544a445619a9e3ceee6af4e6b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:16:20.445302 32253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:20.444922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.130:60219)
0811 05:16:20.445152 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:20.445204 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":71,"threads_started":1}
I20260811 05:16:20.639715 32255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:20.639315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.131:35275)
0811 05:16:20.639565 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:20.639618 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":74,"threads_started":1}
I20260811 05:16:21.281257 32257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:21.280832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.129:60131)
0811 05:16:21.281105 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:21.281157 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":100,"threads_started":1}
I20260811 05:16:21.446419 32259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:21.445982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.130:51047)
0811 05:16:21.446244 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:21.446299 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":77,"threads_started":1}
I20260811 05:16:21.641013 32262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:21.640584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.131:57271)
0811 05:16:21.640852 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:21.640912 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":79,"threads_started":1}
I20260811 05:16:22.282413 32263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:22.281986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.129:58237)
0811 05:16:22.282233 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:22.282312 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":78,"threads_started":1}
I20260811 05:16:22.447690 32264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:22.447239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.130:38449)
0811 05:16:22.447524 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:22.447582 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":98,"threads_started":1}
I20260811 05:16:22.642288 32265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:22.641796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.131:55563)
0811 05:16:22.642134 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:22.642191 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":76,"threads_started":1}
I20260811 05:16:23.283602 32267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:23.283194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.129:52375)
0811 05:16:23.283446 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:23.283500 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":75,"threads_started":1}
I20260811 05:16:23.449033 32268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:23.448620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.130:38083)
0811 05:16:23.448879 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:23.448933 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":78,"threads_started":1}
I20260811 05:16:23.643487 32269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:23.643076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.131:37609)
0811 05:16:23.643335 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:23.643390 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":73,"threads_started":1}
I20260811 05:16:24.285110 32270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:24.284625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.129:45463)
0811 05:16:24.284911 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:24.284981 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":104,"threads_started":1}
I20260811 05:16:24.450285 32271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:24.449837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.130:59701)
0811 05:16:24.450110 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:24.450170 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":76,"threads_started":1}
I20260811 05:16:24.644748 32272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:24.644305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.131:36405)
0811 05:16:24.644604 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:24.644655 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":103,"threads_started":1}
I20260811 05:16:25.286648 32273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:25.286183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.129:43433)
0811 05:16:25.286491 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:25.286551 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":96,"threads_started":1}
I20260811 05:16:25.451761 32274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:25.451315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.130:58981)
0811 05:16:25.451601 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:25.451659 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":91,"threads_started":1}
I20260811 05:16:25.629478 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 31448
I20260811 05:16:25.649447 32275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:25.648953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38739 (local address 127.7.67.131:56117)
0811 05:16:25.649246 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:25.649324 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:38739: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":86,"threads_started":1}
I20260811 05:16:25.656474 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 31732
I20260811 05:16:25.674559 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 31976
2026-08-11T05:16:25Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19678 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:16:25 dist-test-slave-04bb krb5kdc[32287](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:16:25 dist-test-slave-04bb krb5kdc[32287](info): set up 2 sockets
Aug 11 05:16:25 dist-test-slave-04bb krb5kdc[32287](info): commencing operation
krb5kdc: starting...
W20260811 05:16:26.969756 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.264s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:16:27 dist-test-slave-04bb krb5kdc[32287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:16:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:16:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:16:27.103752 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:42737
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:42737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:16:27.221706 32308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:27.221967 32308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:27.222040 32308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:27.226275 32308 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 05:16:27.226336 32308 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 05:16:27.226353 32308 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 05:16:27.226367 32308 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:16:27.226385 32308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:27.226459 32308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:27.226490 32308 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:16:27.226521 32308 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:16:27.231835 32308 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:42737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:42737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:16:27.232949 32308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:27.234028 32308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:27.242817 32316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:27.242816 32318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:27.242800 32315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:27.242996 32308 server_base.cc:1034] running on GCE node
I20260811 05:16:27.243757 32308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:27.245076 32308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:27.246268 32308 hybrid_clock.cc:648] HybridClock initialized: now 1786425387246247 us; error 33 us; skew 500 ppm
Aug 11 05:16:27 dist-test-slave-04bb krb5kdc[32287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425387, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:27.249187 32308 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:16:27.250088 32308 webserver.cc:460] Webserver started at http://127.7.67.190:35313/ using document root <none> and password file <none>
I20260811 05:16:27.250613 32308 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:27.250676 32308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:27.250841 32308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:16:27.252892 32308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "ffae156278c1481c9d8ecebae8f56f94"
format_stamp: "Formatted at 2026-08-11 05:16:27 on dist-test-slave-04bb"
server_key: "08f8e89093750e0a825ee8e46987e228"
server_key_iv: "67368a5d2120912ad7fab1e92fa16432"
server_key_version: "encryptionkey@0"
I20260811 05:16:27.253420 32308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "ffae156278c1481c9d8ecebae8f56f94"
format_stamp: "Formatted at 2026-08-11 05:16:27 on dist-test-slave-04bb"
server_key: "08f8e89093750e0a825ee8e46987e228"
server_key_iv: "67368a5d2120912ad7fab1e92fa16432"
server_key_version: "encryptionkey@0"
I20260811 05:16:27.257032 32308 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:16:27.259390 32328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:27.260706 32308 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:16:27.260838 32308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "ffae156278c1481c9d8ecebae8f56f94"
format_stamp: "Formatted at 2026-08-11 05:16:27 on dist-test-slave-04bb"
server_key: "08f8e89093750e0a825ee8e46987e228"
server_key_iv: "67368a5d2120912ad7fab1e92fa16432"
server_key_version: "encryptionkey@0"
I20260811 05:16:27.260949 32308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:27.280652 32308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:27.283708 32308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:27.283929 32308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:27.291316 32308 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:42737
I20260811 05:16:27.291639 32407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:42737 every 8 connection(s)
I20260811 05:16:27.292315 32308 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:16:27.295260 32408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:27.300796 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 32308
I20260811 05:16:27.300928 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:16:27.301236 7438 external_mini_cluster.cc:1428] Setting key 22d2c2bab95f2420a874c2ce43adc802
I20260811 05:16:27.301385 32408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94: Bootstrap starting.
I20260811 05:16:27.303992 32408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:27.304965 32408 log.cc:824] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:27.307124 32408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94: No bootstrap required, opened a new log
Aug 11 05:16:27 dist-test-slave-04bb krb5kdc[32287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:16:27.311704 32408 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffae156278c1481c9d8ecebae8f56f94" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42737 } }
I20260811 05:16:27.311977 32408 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:27.312079 32408 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffae156278c1481c9d8ecebae8f56f94, State: Initialized, Role: FOLLOWER
I20260811 05:16:27.312611 32408 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffae156278c1481c9d8ecebae8f56f94" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42737 } }
I20260811 05:16:27.312778 32408 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:16:27.312865 32408 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:16:27.312989 32408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:27.313871 32408 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffae156278c1481c9d8ecebae8f56f94" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42737 } }
I20260811 05:16:27.314227 32408 leader_election.cc:302] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffae156278c1481c9d8ecebae8f56f94; no voters:
I20260811 05:16:27.314584 32408 leader_election.cc:288] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:16:27.314658 32414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:16:27.314868 32414 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 1 LEADER]: Becoming Leader. State: Replica: ffae156278c1481c9d8ecebae8f56f94, State: Running, Role: LEADER
I20260811 05:16:27.315227 32414 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffae156278c1481c9d8ecebae8f56f94" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42737 } }
I20260811 05:16:27.315959 32408 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:16:27.317096 32415 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffae156278c1481c9d8ecebae8f56f94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffae156278c1481c9d8ecebae8f56f94" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42737 } } }
I20260811 05:16:27.317129 32419 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffae156278c1481c9d8ecebae8f56f94. Latest consensus state: current_term: 1 leader_uuid: "ffae156278c1481c9d8ecebae8f56f94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffae156278c1481c9d8ecebae8f56f94" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42737 } } }
I20260811 05:16:27.317315 32415 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [sys.catalog]: This master's current role is: LEADER
I20260811 05:16:27.317320 32419 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [sys.catalog]: This master's current role is: LEADER
I20260811 05:16:27.317781 32424 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:16:27.317838 32411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:27.302640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38272 (local address 127.7.67.190:42737)
0811 05:16:27.303064 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:27.303081 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:27.303119 (+ 38us) server_negotiation.cc:407] Connection header received
0811 05:16:27.303773 (+ 654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:27.303790 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:27.304197 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:27.304595 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:27.305513 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:27.306704 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:27.307884 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:27.308362 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:27.308487 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:27.311534 (+ 3047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:27.311540 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:27.311554 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:27.311581 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:27.313659 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:27.314148 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:27.314154 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:27.314161 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:27.314286 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:27.314601 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:27.314605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:27.314607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:27.315186 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:27.315384 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:27.316012 (+ 628us) server_negotiation.cc:299] Negotiation successful
0811 05:16:27.316314 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":134,"threads_started":1}
I20260811 05:16:27.321266 32424 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:16:27.327033 32424 catalog_manager.cc:1347] Generated new cluster ID: bd06e0924a30480ba9d1f9040f938480
I20260811 05:16:27.327096 32424 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:16:27.335577 32424 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:16:27.336488 32424 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:16:27.342056 32424 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94: Generated new TSK 0
I20260811 05:16:27.342706 32424 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
I20260811 05:16:27.380862 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:42737
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:16:27.520368 32449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:27.520736 32449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:27.520826 32449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:27.524830 32449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:27.524925 32449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:27.525017 32449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:16:27.530501 32449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:42737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:16:27.531663 32449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:27.532783 32449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:27.543689 32462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:27.543658 32460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:27.543669 32459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:27.543938 32449 server_base.cc:1034] running on GCE node
I20260811 05:16:27.544489 32449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:27.545221 32449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:27.546399 32449 hybrid_clock.cc:648] HybridClock initialized: now 1786425387546382 us; error 29 us; skew 500 ppm
Aug 11 05:16:27 dist-test-slave-04bb krb5kdc[32287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425387, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:27.549335 32449 init.cc:377] Logged in from keytab as kudu/127.7.67.129@KRBTEST.COM (short username kudu)
I20260811 05:16:27.550197 32449 webserver.cc:460] Webserver started at http://127.7.67.129:43991/ using document root <none> and password file <none>
I20260811 05:16:27.550752 32449 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:27.550810 32449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:27.550983 32449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:16:27.553108 32449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "16a40bffec56478e9afdaf0b92114754"
format_stamp: "Formatted at 2026-08-11 05:16:27 on dist-test-slave-04bb"
server_key: "e466aba3d0cc6274a6f0526862a2242c"
server_key_iv: "da64a65071ae364bcf9ea510c8655afe"
server_key_version: "encryptionkey@0"
I20260811 05:16:27.553639 32449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "16a40bffec56478e9afdaf0b92114754"
format_stamp: "Formatted at 2026-08-11 05:16:27 on dist-test-slave-04bb"
server_key: "e466aba3d0cc6274a6f0526862a2242c"
server_key_iv: "da64a65071ae364bcf9ea510c8655afe"
server_key_version: "encryptionkey@0"
I20260811 05:16:27.557421 32449 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 05:16:27.559975 32472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:27.561335 32449 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 05:16:27.561465 32449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "16a40bffec56478e9afdaf0b92114754"
format_stamp: "Formatted at 2026-08-11 05:16:27 on dist-test-slave-04bb"
server_key: "e466aba3d0cc6274a6f0526862a2242c"
server_key_iv: "da64a65071ae364bcf9ea510c8655afe"
server_key_version: "encryptionkey@0"
I20260811 05:16:27.561573 32449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:27.574757 32449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:27.577730 32449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:27.577939 32449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:27.578562 32449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:27.579411 32449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:16:27.579459 32449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:27.579528 32449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:16:27.579558 32449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:27.588615 32449 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:34045
I20260811 05:16:27.588653 32646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:34045 every 8 connection(s)
I20260811 05:16:27.589543 32449 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
Aug 11 05:16:27 dist-test-slave-04bb krb5kdc[32287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425387, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:16:27.599047 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 32449
I20260811 05:16:27.599180 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:16:27.599494 7438 external_mini_cluster.cc:1428] Setting key ce4c8189fae6485e8cda784248880e06
I20260811 05:16:27.603262 32411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:27.591345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:47621 (local address 127.7.67.190:42737)
0811 05:16:27.591482 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:27.591486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:27.592627 (+ 1141us) server_negotiation.cc:407] Connection header received
0811 05:16:27.593526 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:27.593530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:27.593584 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:27.593674 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:27.594851 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:27.595386 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:27.596561 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:27.596793 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:27.596888 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:27.599240 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:27.599243 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:27.599245 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:27.599326 (+ 81us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:27.601093 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:27.601584 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:27.601590 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:27.601595 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:27.601689 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:27.602024 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:27.602030 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:27.602035 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:27.602290 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:27.602396 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:27.602881 (+ 485us) server_negotiation.cc:299] Negotiation successful
0811 05:16:27.603024 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 05:16:27.604187 32650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:27.591744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.129:47621)
0811 05:16:27.592423 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:27.592465 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:27.593324 (+ 859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:27.593849 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:27.593856 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:27.594217 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:27.594690 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:27.594702 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:27.595495 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:27.595498 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:27.596412 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:27.596422 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:27.596937 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:27.596939 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:27.597078 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:27.597626 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:27.597642 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:27.599059 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:27.601204 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:27.601208 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:27.601221 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:27.601473 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:27.601793 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:27.601795 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:27.601797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:27.601929 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:27.602392 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:27.602397 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:27.602628 (+ 231us) client_negotiation.cc:769] Sending connection context
0811 05:16:27.602797 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 05:16:27.603045 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":120,"threads_started":1}
I20260811 05:16:27.605581 32647 heartbeater.cc:347] Connected to a master server at 127.7.67.190:42737
I20260811 05:16:27.605895 32647 heartbeater.cc:464] Registering TS with master...
I20260811 05:16:27.606520 32647 heartbeater.cc:511] Master 127.7.67.190:42737 requested a full tablet report, sending...
I20260811 05:16:27.608158 32355 ts_manager.cc:194] Registered new tserver with Master: 16a40bffec56478e9afdaf0b92114754 (127.7.67.129:34045)
I20260811 05:16:27.609344 32355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.129@KRBTEST.COM'} at 127.7.67.129:47621
WARNING: no policy specified for kudu/127.7.67.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
I20260811 05:16:27.632854 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:42737
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:16:27.758507 32656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:27.758785 32656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:27.758870 32656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:27.762979 32656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:27.763041 32656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:27.763154 32656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:16:27.768846 32656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:42737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32656
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:16:27.770008 32656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:27.771399 32656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:27.781731 32662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:27.781774 32665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:27.781774 32656 server_base.cc:1034] running on GCE node
W20260811 05:16:27.781731 32669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:27.782447 32656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:27.783124 32656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:27.784296 32656 hybrid_clock.cc:648] HybridClock initialized: now 1786425387784281 us; error 30 us; skew 500 ppm
Aug 11 05:16:27 dist-test-slave-04bb krb5kdc[32287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425387, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:27.787212 32656 init.cc:377] Logged in from keytab as kudu/127.7.67.130@KRBTEST.COM (short username kudu)
I20260811 05:16:27.788120 32656 webserver.cc:460] Webserver started at http://127.7.67.130:34309/ using document root <none> and password file <none>
I20260811 05:16:27.788754 32656 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:27.788818 32656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:27.789057 32656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:16:27.791112 32656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "3ec2f199c1844981a74ee202ad6d2195"
format_stamp: "Formatted at 2026-08-11 05:16:27 on dist-test-slave-04bb"
server_key: "038e26bd0d9f3e70ab8f08d2d0b09a4f"
server_key_iv: "a4ac55458dfdc7c26b2f86c844e2bf65"
server_key_version: "encryptionkey@0"
I20260811 05:16:27.791687 32656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "3ec2f199c1844981a74ee202ad6d2195"
format_stamp: "Formatted at 2026-08-11 05:16:27 on dist-test-slave-04bb"
server_key: "038e26bd0d9f3e70ab8f08d2d0b09a4f"
server_key_iv: "a4ac55458dfdc7c26b2f86c844e2bf65"
server_key_version: "encryptionkey@0"
I20260811 05:16:27.795476 32656 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 05:16:27.797982 32682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:27.799288 32656 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:16:27.799445 32656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "3ec2f199c1844981a74ee202ad6d2195"
format_stamp: "Formatted at 2026-08-11 05:16:27 on dist-test-slave-04bb"
server_key: "038e26bd0d9f3e70ab8f08d2d0b09a4f"
server_key_iv: "a4ac55458dfdc7c26b2f86c844e2bf65"
server_key_version: "encryptionkey@0"
I20260811 05:16:27.799569 32656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:27.822629 32656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:27.826143 32656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:27.826362 32656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:27.826984 32656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:27.827911 32656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:16:27.827986 32656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:27.828058 32656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:16:27.828109 32656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:27.836894 32656 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:38701
I20260811 05:16:27.836922 386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:38701 every 8 connection(s)
I20260811 05:16:27.837808 32656 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:16:27.841446 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 32656
I20260811 05:16:27.841581 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:16:27.841938 7438 external_mini_cluster.cc:1428] Setting key 29a40c9727b5145a81a522f8fa9ab065
Aug 11 05:16:27 dist-test-slave-04bb krb5kdc[32287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425387, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.131@KRBTEST.COM; defaulting to no policy
I20260811 05:16:27.853580 32411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:27.839591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:48565 (local address 127.7.67.190:42737)
0811 05:16:27.839758 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:27.839763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:27.840716 (+ 953us) server_negotiation.cc:407] Connection header received
0811 05:16:27.842349 (+ 1633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:27.842353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:27.842401 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:27.842466 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:27.844011 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:27.844856 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:27.846474 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:27.846689 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:27.846769 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:27.849404 (+ 2635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:27.849410 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:27.849417 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:27.849453 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:27.851243 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:27.851784 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:27.851788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:27.851790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:27.851883 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:27.852332 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:27.852339 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:27.852340 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:27.852644 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:27.852756 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:27.853353 (+ 597us) server_negotiation.cc:299] Negotiation successful
0811 05:16:27.853455 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 05:16:27.854470 394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:27.839963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.130:48565)
0811 05:16:27.840497 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:27.840599 (+ 102us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:27.842076 (+ 1477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:27.842801 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:27.842807 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:27.843254 (+ 447us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:27.843859 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:27.843873 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:27.844984 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:27.844987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:27.846337 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:27.846352 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:27.846851 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:27.846858 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:27.847085 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:27.847699 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:27.847715 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:27.849225 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:27.851377 (+ 2152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:27.851382 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:27.851395 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:27.851665 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:27.852034 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:27.852037 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:27.852039 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:27.852207 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:27.852772 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:27.852777 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:27.853036 (+ 259us) client_negotiation.cc:769] Sending connection context
0811 05:16:27.853267 (+ 231us) client_negotiation.cc:240] Negotiation successful
0811 05:16:27.853532 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":112,"threads_started":1}
I20260811 05:16:27.855691 390 heartbeater.cc:347] Connected to a master server at 127.7.67.190:42737
I20260811 05:16:27.855926 390 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.131@KRBTEST.COM" created.
I20260811 05:16:27.856487 390 heartbeater.cc:511] Master 127.7.67.190:42737 requested a full tablet report, sending...
I20260811 05:16:27.857740 32355 ts_manager.cc:194] Registered new tserver with Master: 3ec2f199c1844981a74ee202ad6d2195 (127.7.67.130:38701)
I20260811 05:16:27.858357 32355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.130@KRBTEST.COM'} at 127.7.67.130:48565
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
I20260811 05:16:27.873677 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:42737
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:16:27.989061 397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:27.989320 397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:27.989362 397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:27.993373 397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:27.993431 397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:27.993489 397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:16:27.998575 397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:42737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:16:27.999604 397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:28.000698 397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:16:28.011749 397 server_base.cc:1034] running on GCE node
W20260811 05:16:28.011765 404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:28.012051 403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:28.012269 407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:28.012794 397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:28.013542 397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:28.014722 397 hybrid_clock.cc:648] HybridClock initialized: now 1786425388014706 us; error 27 us; skew 500 ppm
Aug 11 05:16:28 dist-test-slave-04bb krb5kdc[32287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425388, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:28.018055 397 init.cc:377] Logged in from keytab as kudu/127.7.67.131@KRBTEST.COM (short username kudu)
I20260811 05:16:28.019078 397 webserver.cc:460] Webserver started at http://127.7.67.131:43077/ using document root <none> and password file <none>
I20260811 05:16:28.019728 397 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:28.019833 397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:28.020102 397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:16:28.022401 397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "5dbd9f0248d848c5933483a5516ed43c"
format_stamp: "Formatted at 2026-08-11 05:16:28 on dist-test-slave-04bb"
server_key: "5db8d961ab4b40408ab8bdbf23fdefc3"
server_key_iv: "24b98f35ecae6c11c1a2f3e584947c2a"
server_key_version: "encryptionkey@0"
I20260811 05:16:28.023034 397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "5dbd9f0248d848c5933483a5516ed43c"
format_stamp: "Formatted at 2026-08-11 05:16:28 on dist-test-slave-04bb"
server_key: "5db8d961ab4b40408ab8bdbf23fdefc3"
server_key_iv: "24b98f35ecae6c11c1a2f3e584947c2a"
server_key_version: "encryptionkey@0"
I20260811 05:16:28.027097 397 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260811 05:16:28.029809 420 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:28.031240 397 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 05:16:28.031392 397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "5dbd9f0248d848c5933483a5516ed43c"
format_stamp: "Formatted at 2026-08-11 05:16:28 on dist-test-slave-04bb"
server_key: "5db8d961ab4b40408ab8bdbf23fdefc3"
server_key_iv: "24b98f35ecae6c11c1a2f3e584947c2a"
server_key_version: "encryptionkey@0"
I20260811 05:16:28.031509 397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:28.053815 397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:28.057279 397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:28.057492 397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:28.058107 397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:28.058965 397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:16:28.059036 397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:28.059108 397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:16:28.059219 397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:28.068719 397 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:39339
I20260811 05:16:28.068750 604 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:39339 every 8 connection(s)
I20260811 05:16:28.069646 397 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:16:28.071206 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 397
I20260811 05:16:28.071337 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:16:28.071681 7438 external_mini_cluster.cc:1428] Setting key 7792f34b81616a6aa092979509d7c5e9
Aug 11 05:16:28 dist-test-slave-04bb krb5kdc[32287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425388, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:16:28.084023 32411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:28.071568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:42033 (local address 127.7.67.190:42737)
0811 05:16:28.071738 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:28.071743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:28.072938 (+ 1195us) server_negotiation.cc:407] Connection header received
0811 05:16:28.074523 (+ 1585us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:28.074526 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:28.074571 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:28.074638 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:28.075742 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.076252 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.077451 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.077663 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.077743 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:28.080296 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:28.080299 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:28.080301 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:28.080325 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:28.082020 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:28.082565 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:28.082568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:28.082570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:28.082669 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:28.083012 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:28.083015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:28.083016 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:28.083209 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:28.083289 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:28.083788 (+ 499us) server_negotiation.cc:299] Negotiation successful
0811 05:16:28.083891 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 05:16:28.085144 608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:28.072072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.131:42033)
0811 05:16:28.072791 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:28.072844 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:28.074333 (+ 1489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:28.074772 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:28.074778 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:28.075123 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:28.075618 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:28.075632 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.076377 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.076380 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:28.077339 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:28.077349 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.077792 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.077794 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:28.077935 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:28.078600 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:28.078617 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:28.080111 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:28.082144 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:28.082149 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:28.082162 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:28.082457 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:28.082770 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:28.082773 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:28.082774 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:28.082921 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:28.083309 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:28.083314 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:28.083551 (+ 237us) client_negotiation.cc:769] Sending connection context
0811 05:16:28.083722 (+ 171us) client_negotiation.cc:240] Negotiation successful
0811 05:16:28.084002 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":164,"threads_started":1}
I20260811 05:16:28.086320 605 heartbeater.cc:347] Connected to a master server at 127.7.67.190:42737
I20260811 05:16:28.086527 605 heartbeater.cc:464] Registering TS with master...
I20260811 05:16:28.086975 605 heartbeater.cc:511] Master 127.7.67.190:42737 requested a full tablet report, sending...
I20260811 05:16:28.088002 32355 ts_manager.cc:194] Registered new tserver with Master: 5dbd9f0248d848c5933483a5516ed43c (127.7.67.131:39339)
I20260811 05:16:28.088618 32355 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.131@KRBTEST.COM'} at 127.7.67.131:42033
I20260811 05:16:28.098348 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:16:28.108687 32411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:28.100915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38280 (local address 127.7.67.190:42737)
0811 05:16:28.101046 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:28.101053 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:28.101213 (+ 160us) server_negotiation.cc:407] Connection header received
0811 05:16:28.101374 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:28.101377 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:28.101413 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:28.101481 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:28.102248 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.102771 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.103729 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.103991 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.104068 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:28.105669 (+ 1601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:28.105672 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:28.105674 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:28.105694 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:28.107085 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:28.107514 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:28.107517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:28.107518 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:28.107597 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:28.107849 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:28.107851 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:28.107852 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:28.108065 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:28.108169 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:28.108418 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 05:16:28.108508 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:16:28.111515 32355 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:16:28.114579 32355 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:16:28.129467 620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:28.123520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:39339 (local address 127.0.0.1:40276)
0811 05:16:28.124401 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:28.124472 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:28.124637 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:28.125460 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:28.125465 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:28.125483 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:28.125644 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:28.125652 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.126775 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.126779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:28.127999 (+ 1220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:28.128010 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.128950 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.128953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:28.129058 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:28.129098 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 05:16:28.129208 (+ 110us) client_negotiation.cc:240] Negotiation successful
0811 05:16:28.129319 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"thread_start_us":94,"threads_started":1}
I20260811 05:16:28.129818 624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:28.123733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40276 (local address 127.7.67.131:39339)
0811 05:16:28.124610 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:28.124625 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:28.124655 (+ 30us) server_negotiation.cc:407] Connection header received
0811 05:16:28.124864 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:28.124874 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:28.125102 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:28.125264 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:28.125781 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.126641 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.128114 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.128836 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.128952 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:28.129095 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:28.129265 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 05:16:28.129366 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":730,"thread_start_us":145,"threads_started":1}
I20260811 05:16:28.129977 627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:28.124517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:38701 (local address 127.0.0.1:35556)
0811 05:16:28.124807 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:28.124821 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:28.124915 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:28.125662 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:28.125668 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:28.125692 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:28.125796 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:28.125803 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.127253 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.127257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:28.128450 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:28.128461 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.129570 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.129576 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:28.129671 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:28.129706 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 05:16:28.129754 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 05:16:28.129820 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"mutex_wait_us":62,"thread_start_us":103,"threads_started":1}
I20260811 05:16:28.130431 621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:28.123728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:34045 (local address 127.0.0.1:33970)
0811 05:16:28.125231 (+ 1503us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:28.125246 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:28.125338 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:28.125872 (+ 534us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:28.125874 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:28.125893 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:28.125967 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:28.125973 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.127547 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.127552 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:28.128741 (+ 1189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:28.128749 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.130020 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.130023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:28.130138 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:28.130160 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 05:16:28.130234 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 05:16:28.130311 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":836,"mutex_wait_us":1,"thread_start_us":103,"threads_started":1}
I20260811 05:16:28.130697 629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:28.124734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35556 (local address 127.7.67.130:38701)
0811 05:16:28.125190 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:28.125204 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:28.125229 (+ 25us) server_negotiation.cc:407] Connection header received
0811 05:16:28.125316 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:28.125321 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:28.125512 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:28.125666 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:28.125930 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.127107 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.128609 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.129423 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.130240 (+ 817us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:28.130387 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:28.130435 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 05:16:28.130515 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":175,"threads_started":1}
I20260811 05:16:28.130913 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:28.123910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33970 (local address 127.7.67.129:34045)
0811 05:16:28.124239 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:28.124246 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:28.125339 (+ 1093us) server_negotiation.cc:407] Connection header received
0811 05:16:28.125435 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:28.125447 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:28.125592 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:28.125756 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:28.126096 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.127210 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.128869 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.129684 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.129790 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:28.129956 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:28.130271 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 05:16:28.130369 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":63,"threads_started":1}
I20260811 05:16:28.132871 496 tablet_service.cc:1467] Processing CreateTablet for tablet 77c8fdbed0c24a09bc57627e6e996528 (DEFAULT_TABLE table=test-table [id=652ec223cc9f495cbb644bd4d5c3bc74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:16:28.133473 32550 tablet_service.cc:1467] Processing CreateTablet for tablet 77c8fdbed0c24a09bc57627e6e996528 (DEFAULT_TABLE table=test-table [id=652ec223cc9f495cbb644bd4d5c3bc74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:16:28.133479 32755 tablet_service.cc:1467] Processing CreateTablet for tablet 77c8fdbed0c24a09bc57627e6e996528 (DEFAULT_TABLE table=test-table [id=652ec223cc9f495cbb644bd4d5c3bc74]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:16:28.133916 496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77c8fdbed0c24a09bc57627e6e996528. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:28.134528 32550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77c8fdbed0c24a09bc57627e6e996528. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:28.134528 32755 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 77c8fdbed0c24a09bc57627e6e996528. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:28.140712 630 tablet_bootstrap.cc:492] T 77c8fdbed0c24a09bc57627e6e996528 P 5dbd9f0248d848c5933483a5516ed43c: Bootstrap starting.
I20260811 05:16:28.142299 632 tablet_bootstrap.cc:492] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195: Bootstrap starting.
I20260811 05:16:28.142367 631 tablet_bootstrap.cc:492] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754: Bootstrap starting.
I20260811 05:16:28.143208 630 tablet_bootstrap.cc:654] T 77c8fdbed0c24a09bc57627e6e996528 P 5dbd9f0248d848c5933483a5516ed43c: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:28.144145 631 tablet_bootstrap.cc:654] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:28.144515 632 tablet_bootstrap.cc:654] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:28.144650 630 log.cc:824] T 77c8fdbed0c24a09bc57627e6e996528 P 5dbd9f0248d848c5933483a5516ed43c: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:28.145027 631 log.cc:824] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:28.145893 632 log.cc:824] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:28.147125 631 tablet_bootstrap.cc:492] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754: No bootstrap required, opened a new log
I20260811 05:16:28.147243 630 tablet_bootstrap.cc:492] T 77c8fdbed0c24a09bc57627e6e996528 P 5dbd9f0248d848c5933483a5516ed43c: No bootstrap required, opened a new log
I20260811 05:16:28.147352 631 ts_tablet_manager.cc:1397] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 05:16:28.147432 630 ts_tablet_manager.cc:1397] T 77c8fdbed0c24a09bc57627e6e996528 P 5dbd9f0248d848c5933483a5516ed43c: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 05:16:28.147830 632 tablet_bootstrap.cc:492] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195: No bootstrap required, opened a new log
I20260811 05:16:28.148018 632 ts_tablet_manager.cc:1397] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 05:16:28.150954 631 raft_consensus.cc:348] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } }
I20260811 05:16:28.151201 631 raft_consensus.cc:374] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:28.151254 631 raft_consensus.cc:729] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16a40bffec56478e9afdaf0b92114754, State: Initialized, Role: FOLLOWER
I20260811 05:16:28.151392 630 raft_consensus.cc:348] T 77c8fdbed0c24a09bc57627e6e996528 P 5dbd9f0248d848c5933483a5516ed43c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } }
I20260811 05:16:28.151659 630 raft_consensus.cc:374] T 77c8fdbed0c24a09bc57627e6e996528 P 5dbd9f0248d848c5933483a5516ed43c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:28.151746 630 raft_consensus.cc:729] T 77c8fdbed0c24a09bc57627e6e996528 P 5dbd9f0248d848c5933483a5516ed43c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dbd9f0248d848c5933483a5516ed43c, State: Initialized, Role: FOLLOWER
I20260811 05:16:28.151757 631 consensus_queue.cc:260] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } }
I20260811 05:16:28.151729 632 raft_consensus.cc:348] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } }
I20260811 05:16:28.152002 632 raft_consensus.cc:374] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:28.152060 632 raft_consensus.cc:729] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ec2f199c1844981a74ee202ad6d2195, State: Initialized, Role: FOLLOWER
I20260811 05:16:28.152225 630 consensus_queue.cc:260] T 77c8fdbed0c24a09bc57627e6e996528 P 5dbd9f0248d848c5933483a5516ed43c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } }
I20260811 05:16:28.152570 632 consensus_queue.cc:260] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } }
I20260811 05:16:28.153004 631 ts_tablet_manager.cc:1428] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 05:16:28.153215 630 ts_tablet_manager.cc:1428] T 77c8fdbed0c24a09bc57627e6e996528 P 5dbd9f0248d848c5933483a5516ed43c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 05:16:28.153339 605 heartbeater.cc:503] Master 127.7.67.190:42737 was elected leader, sending a full tablet report...
I20260811 05:16:28.153394 632 ts_tablet_manager.cc:1428] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 05:16:28.153726 390 heartbeater.cc:503] Master 127.7.67.190:42737 was elected leader, sending a full tablet report...
I20260811 05:16:28.154508 32647 heartbeater.cc:503] Master 127.7.67.190:42737 was elected leader, sending a full tablet report...
I20260811 05:16:28.186820 643 raft_consensus.cc:482] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:16:28.187052 643 raft_consensus.cc:504] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } }
I20260811 05:16:28.188344 643 leader_election.cc:288] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5dbd9f0248d848c5933483a5516ed43c (127.7.67.131:39339), 16a40bffec56478e9afdaf0b92114754 (127.7.67.129:34045)
I20260811 05:16:28.192718 394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:28.188556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:39339 (local address 127.7.67.130:60993)
0811 05:16:28.188753 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:28.188770 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:28.188880 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:28.189198 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:28.189200 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:28.189245 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:28.189405 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:28.189412 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.190301 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.190306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:28.191389 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:28.191401 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.192163 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.192168 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:28.192329 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:28.192369 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 05:16:28.192447 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 05:16:28.192565 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":1}
I20260811 05:16:28.192729 624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:28.188694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:60993 (local address 127.7.67.131:39339)
0811 05:16:28.188821 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:28.188825 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:28.188840 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:16:28.189016 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:28.189021 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:28.189082 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:28.189165 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:28.189558 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.190157 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.191540 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.192028 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.192124 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:28.192170 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:28.192482 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 05:16:28.192584 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 05:16:28.192970 625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:28.188938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:45793 (local address 127.7.67.129:34045)
0811 05:16:28.189067 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:28.189073 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:28.189245 (+ 172us) server_negotiation.cc:407] Connection header received
0811 05:16:28.189384 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:28.189387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:28.189440 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:28.189543 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:28.189847 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.190584 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.191779 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.192303 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.192437 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:28.192485 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:28.192738 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 05:16:28.192829 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:16:28.193068 644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:28.188795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:34045 (local address 127.7.67.130:45793)
0811 05:16:28.189151 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:28.189165 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:28.189269 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:28.189562 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:28.189565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:28.189583 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:28.189694 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:28.189700 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.190742 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.190745 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:28.191642 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:28.191653 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.192448 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.192453 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:28.192593 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:28.192618 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 05:16:28.192695 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 05:16:28.192772 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":39,"spinlock_wait_cycles":9088,"thread_start_us":133,"threads_started":1}
I20260811 05:16:28.193397 533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77c8fdbed0c24a09bc57627e6e996528" candidate_uuid: "3ec2f199c1844981a74ee202ad6d2195" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dbd9f0248d848c5933483a5516ed43c" is_pre_election: true
I20260811 05:16:28.193653 32585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77c8fdbed0c24a09bc57627e6e996528" candidate_uuid: "3ec2f199c1844981a74ee202ad6d2195" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16a40bffec56478e9afdaf0b92114754" is_pre_election: true
I20260811 05:16:28.193755 533 raft_consensus.cc:2454] T 77c8fdbed0c24a09bc57627e6e996528 P 5dbd9f0248d848c5933483a5516ed43c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ec2f199c1844981a74ee202ad6d2195 in term 0.
I20260811 05:16:28.193970 32585 raft_consensus.cc:2454] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ec2f199c1844981a74ee202ad6d2195 in term 0.
I20260811 05:16:28.194361 32683 leader_election.cc:302] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ec2f199c1844981a74ee202ad6d2195, 5dbd9f0248d848c5933483a5516ed43c; no voters:
I20260811 05:16:28.194669 643 raft_consensus.cc:2781] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:16:28.194756 643 raft_consensus.cc:482] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:16:28.194801 643 raft_consensus.cc:3037] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:28.195871 643 raft_consensus.cc:504] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } }
I20260811 05:16:28.196246 643 leader_election.cc:288] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195 [CANDIDATE]: Term 1 election: Requested vote from peers 5dbd9f0248d848c5933483a5516ed43c (127.7.67.131:39339), 16a40bffec56478e9afdaf0b92114754 (127.7.67.129:34045)
I20260811 05:16:28.196669 32585 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77c8fdbed0c24a09bc57627e6e996528" candidate_uuid: "3ec2f199c1844981a74ee202ad6d2195" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16a40bffec56478e9afdaf0b92114754"
I20260811 05:16:28.196691 533 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "77c8fdbed0c24a09bc57627e6e996528" candidate_uuid: "3ec2f199c1844981a74ee202ad6d2195" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dbd9f0248d848c5933483a5516ed43c"
I20260811 05:16:28.196827 533 raft_consensus.cc:3037] T 77c8fdbed0c24a09bc57627e6e996528 P 5dbd9f0248d848c5933483a5516ed43c [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:28.196805 32585 raft_consensus.cc:3037] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:28.197899 533 raft_consensus.cc:2454] T 77c8fdbed0c24a09bc57627e6e996528 P 5dbd9f0248d848c5933483a5516ed43c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ec2f199c1844981a74ee202ad6d2195 in term 1.
I20260811 05:16:28.198163 32585 raft_consensus.cc:2454] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ec2f199c1844981a74ee202ad6d2195 in term 1.
I20260811 05:16:28.198284 32683 leader_election.cc:302] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ec2f199c1844981a74ee202ad6d2195, 5dbd9f0248d848c5933483a5516ed43c; no voters:
I20260811 05:16:28.198444 643 raft_consensus.cc:2781] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:16:28.198705 643 raft_consensus.cc:686] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195 [term 1 LEADER]: Becoming Leader. State: Replica: 3ec2f199c1844981a74ee202ad6d2195, State: Running, Role: LEADER
I20260811 05:16:28.199060 643 consensus_queue.cc:237] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } }
I20260811 05:16:28.202549 32354 catalog_manager.cc:5495] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ec2f199c1844981a74ee202ad6d2195 (127.7.67.130). New cstate: current_term: 1 leader_uuid: "3ec2f199c1844981a74ee202ad6d2195" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } health_report { overall_health: HEALTHY } } }
I20260811 05:16:28.228348 629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:28.224995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35560 (local address 127.7.67.130:38701)
0811 05:16:28.225137 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:28.225140 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:28.225154 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:16:28.225233 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:28.225235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:28.225290 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:28.225381 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:16:28.225608 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.226111 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.226935 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:28.227140 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:28.227218 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:28.227487 (+ 269us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:16:28.227940 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:16:28.228033 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:28.228094 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 05:16:28.228144 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:16:28.234728 533 raft_consensus.cc:1261] T 77c8fdbed0c24a09bc57627e6e996528 P 5dbd9f0248d848c5933483a5516ed43c [term 1 FOLLOWER]: Refusing update from remote peer 3ec2f199c1844981a74ee202ad6d2195: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:16:28.234727 32585 raft_consensus.cc:1261] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Refusing update from remote peer 3ec2f199c1844981a74ee202ad6d2195: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:16:28.235487 643 consensus_queue.cc:1035] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:16:28.235711 645 consensus_queue.cc:1035] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:16:28.244374 653 mvcc.cc:205] Tried to move back new op lower bound from 7317198390201356288 to 7317198390066458624. Current Snapshot: MvccSnapshot[applied={T|T < 7317198390201356288}]
I20260811 05:16:28.252563 32355 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:16:28.252744 32355 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:16:28.255287 32355 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94: Sending DeleteTablet for 3 replicas of tablet 77c8fdbed0c24a09bc57627e6e996528
I20260811 05:16:28.256089 496 tablet_service.cc:1514] Processing DeleteTablet for tablet 77c8fdbed0c24a09bc57627e6e996528 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:16:28 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:40276
I20260811 05:16:28.256089 32550 tablet_service.cc:1514] Processing DeleteTablet for tablet 77c8fdbed0c24a09bc57627e6e996528 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:16:28 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:33970
I20260811 05:16:28.256281 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 32449
I20260811 05:16:28.256738 32755 tablet_service.cc:1514] Processing DeleteTablet for tablet 77c8fdbed0c24a09bc57627e6e996528 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:16:28 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:35556
I20260811 05:16:28.259331 662 tablet_replica.cc:330] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195: stopping tablet replica
I20260811 05:16:28.259580 661 tablet_replica.cc:330] T 77c8fdbed0c24a09bc57627e6e996528 P 5dbd9f0248d848c5933483a5516ed43c: stopping tablet replica
I20260811 05:16:28.259680 662 raft_consensus.cc:2229] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:16:28.259972 661 raft_consensus.cc:2229] T 77c8fdbed0c24a09bc57627e6e996528 P 5dbd9f0248d848c5933483a5516ed43c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:16:28.260124 662 raft_consensus.cc:2258] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:16:28.260406 661 raft_consensus.cc:2258] T 77c8fdbed0c24a09bc57627e6e996528 P 5dbd9f0248d848c5933483a5516ed43c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:16:28.263382 662 ts_tablet_manager.cc:1905] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:16:28.263382 661 ts_tablet_manager.cc:1905] T 77c8fdbed0c24a09bc57627e6e996528 P 5dbd9f0248d848c5933483a5516ed43c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:16:28.266367 661 ts_tablet_manager.cc:1918] T 77c8fdbed0c24a09bc57627e6e996528 P 5dbd9f0248d848c5933483a5516ed43c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:16:28.266367 662 ts_tablet_manager.cc:1918] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:16:28.266474 662 log.cc:1196] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/77c8fdbed0c24a09bc57627e6e996528
I20260811 05:16:28.266594 661 log.cc:1196] T 77c8fdbed0c24a09bc57627e6e996528 P 5dbd9f0248d848c5933483a5516ed43c: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/77c8fdbed0c24a09bc57627e6e996528
I20260811 05:16:28.266855 662 ts_tablet_manager.cc:1939] T 77c8fdbed0c24a09bc57627e6e996528 P 3ec2f199c1844981a74ee202ad6d2195: Deleting consensus metadata
I20260811 05:16:28.267046 661 ts_tablet_manager.cc:1939] T 77c8fdbed0c24a09bc57627e6e996528 P 5dbd9f0248d848c5933483a5516ed43c: Deleting consensus metadata
I20260811 05:16:28.268190 32329 catalog_manager.cc:4841] TS 5dbd9f0248d848c5933483a5516ed43c (127.7.67.131:39339): tablet 77c8fdbed0c24a09bc57627e6e996528 (table test-table [id=652ec223cc9f495cbb644bd4d5c3bc74]) successfully deleted
I20260811 05:16:28.290640 32333 catalog_manager.cc:4841] TS 3ec2f199c1844981a74ee202ad6d2195 (127.7.67.130:38701): tablet 77c8fdbed0c24a09bc57627e6e996528 (table test-table [id=652ec223cc9f495cbb644bd4d5c3bc74]) successfully deleted
W20260811 05:16:28.298548 32331 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.129:34045 (error 108)
I20260811 05:16:28.299499 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 32656
I20260811 05:16:28.299729 621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:28.299407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:34045 (local address 127.0.0.1:33976)
0811 05:16:28.299529 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:28.299611 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.129:34045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 05:16:28.302284 32331 catalog_manager.cc:4584] TS 16a40bffec56478e9afdaf0b92114754 (127.7.67.129:34045): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 77c8fdbed0c24a09bc57627e6e996528: Network error: Client connection negotiation failed: client connection to 127.7.67.129:34045: connect: Connection refused (error 111)
I20260811 05:16:28.319332 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 397
I20260811 05:16:28.334170 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 32308
I20260811 05:16:48.348489 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:42737
--webserver_interface=127.7.67.190
--webserver_port=35313
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:42737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:16:48.465915 664 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:48.466182 664 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:48.466251 664 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:48.470348 664 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 05:16:48.470435 664 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 05:16:48.470470 664 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 05:16:48.470494 664 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:16:48.470532 664 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:48.470577 664 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:48.470633 664 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:16:48.470669 664 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:16:48.476480 664 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:42737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:42737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=35313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.664
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:16:48.477625 664 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:48.478646 664 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:48.493391 678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:48.493399 677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:48.493464 664 server_base.cc:1034] running on GCE node
W20260811 05:16:48.493407 681 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:48.494433 664 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:48.495642 664 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:48.496809 664 hybrid_clock.cc:648] HybridClock initialized: now 1786425408496793 us; error 29 us; skew 500 ppm
Aug 11 05:16:48 dist-test-slave-04bb krb5kdc[32287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425408, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:48.505609 664 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:16:48.506728 664 webserver.cc:460] Webserver started at http://127.7.67.190:35313/ using document root <none> and password file <none>
I20260811 05:16:48.507376 664 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:48.507459 664 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:48.511744 664 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 05:16:48.514025 693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:48.515373 664 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:16:48.515522 664 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "ffae156278c1481c9d8ecebae8f56f94"
format_stamp: "Formatted at 2026-08-11 05:16:27 on dist-test-slave-04bb"
server_key: "08f8e89093750e0a825ee8e46987e228"
server_key_iv: "67368a5d2120912ad7fab1e92fa16432"
server_key_version: "encryptionkey@0"
I20260811 05:16:48.516131 664 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:48.547127 664 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:48.550736 664 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:48.550954 664 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:48.558784 664 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:42737
I20260811 05:16:48.559094 764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:42737 every 8 connection(s)
I20260811 05:16:48.559975 664 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:16:48.564699 768 sys_catalog.cc:263] Verifying existing consensus state
I20260811 05:16:48.566452 768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94: Bootstrap starting.
I20260811 05:16:48.567214 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 664
I20260811 05:16:48.567693 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:34045
--local_ip_for_outbound_sockets=127.7.67.129
--tserver_master_addrs=127.7.67.190:42737
--webserver_port=43991
--webserver_interface=127.7.67.129
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:16:48.584493 768 log.cc:824] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:48.593664 768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:16:48.594050 768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94: Bootstrap complete.
I20260811 05:16:48.598328 768 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffae156278c1481c9d8ecebae8f56f94" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42737 } }
I20260811 05:16:48.598748 768 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffae156278c1481c9d8ecebae8f56f94, State: Initialized, Role: FOLLOWER
I20260811 05:16:48.599323 768 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffae156278c1481c9d8ecebae8f56f94" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42737 } }
I20260811 05:16:48.599474 768 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:16:48.599570 768 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:16:48.599707 768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 1 FOLLOWER]: Advancing to term 2
I20260811 05:16:48.601747 768 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffae156278c1481c9d8ecebae8f56f94" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42737 } }
I20260811 05:16:48.602090 768 leader_election.cc:302] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffae156278c1481c9d8ecebae8f56f94; no voters:
I20260811 05:16:48.602449 768 leader_election.cc:288] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 05:16:48.602532 774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 05:16:48.602749 774 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 2 LEADER]: Becoming Leader. State: Replica: ffae156278c1481c9d8ecebae8f56f94, State: Running, Role: LEADER
I20260811 05:16:48.603044 774 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffae156278c1481c9d8ecebae8f56f94" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42737 } }
I20260811 05:16:48.603746 768 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:16:48.604417 775 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffae156278c1481c9d8ecebae8f56f94. Latest consensus state: current_term: 2 leader_uuid: "ffae156278c1481c9d8ecebae8f56f94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffae156278c1481c9d8ecebae8f56f94" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42737 } } }
I20260811 05:16:48.604871 775 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [sys.catalog]: This master's current role is: LEADER
I20260811 05:16:48.604765 776 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ffae156278c1481c9d8ecebae8f56f94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffae156278c1481c9d8ecebae8f56f94" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42737 } } }
I20260811 05:16:48.605257 776 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [sys.catalog]: This master's current role is: LEADER
I20260811 05:16:48.605578 790 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:16:48.610589 790 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:16:48.611218 790 catalog_manager.cc:1259] Loaded cluster ID: bd06e0924a30480ba9d1f9040f938480
I20260811 05:16:48.611279 790 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:16:48.613765 790 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:16:48.614444 790 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94: Loaded TSK: 0
I20260811 05:16:48.633356 790 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94: Generated new TSK 1
I20260811 05:16:48.633625 790 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 05:16:48.691601 772 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:48.691830 772 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:48.691872 772 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:48.695854 772 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:48.695909 772 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:48.695966 772 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:16:48.701349 772 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:34045
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=43991
--tserver_master_addrs=127.7.67.190:42737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.772
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:16:48.702353 772 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:48.703368 772 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:48.714807 814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:48.714852 811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:48.714988 812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:48.715739 772 server_base.cc:1034] running on GCE node
I20260811 05:16:48.716174 772 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:48.716929 772 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:48.718132 772 hybrid_clock.cc:648] HybridClock initialized: now 1786425408718093 us; error 58 us; skew 500 ppm
Aug 11 05:16:48 dist-test-slave-04bb krb5kdc[32287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425408, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:48.721328 772 init.cc:377] Logged in from keytab as kudu/127.7.67.129@KRBTEST.COM (short username kudu)
I20260811 05:16:48.722375 772 webserver.cc:460] Webserver started at http://127.7.67.129:43991/ using document root <none> and password file <none>
I20260811 05:16:48.722968 772 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:48.723079 772 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:48.727208 772 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:16:48.730629 824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:48.731983 772 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.003s
I20260811 05:16:48.732110 772 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "16a40bffec56478e9afdaf0b92114754"
format_stamp: "Formatted at 2026-08-11 05:16:27 on dist-test-slave-04bb"
server_key: "e466aba3d0cc6274a6f0526862a2242c"
server_key_iv: "da64a65071ae364bcf9ea510c8655afe"
server_key_version: "encryptionkey@0"
I20260811 05:16:48.732726 772 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:48.743151 772 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:48.746307 772 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:48.746502 772 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:48.747149 772 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:48.748338 834 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 05:16:48.752509 772 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 05:16:48.752617 772 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.001s
I20260811 05:16:48.752668 772 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 05:16:48.754312 772 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 05:16:48.754371 772 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 05:16:48.754637 834 tablet_bootstrap.cc:492] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754: Bootstrap starting.
I20260811 05:16:48.765633 772 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:34045
I20260811 05:16:48.765745 1001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:34045 every 8 connection(s)
I20260811 05:16:48.766898 772 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:16:48.775254 834 log.cc:824] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:48.775599 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 772
I20260811 05:16:48.776165 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:38701
--local_ip_for_outbound_sockets=127.7.67.130
--tserver_master_addrs=127.7.67.190:42737
--webserver_port=34309
--webserver_interface=127.7.67.130
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:16:48 dist-test-slave-04bb krb5kdc[32287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425408, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:16:48.784051 834 tablet_bootstrap.cc:492] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:16:48.784500 834 tablet_bootstrap.cc:492] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754: Bootstrap complete.
I20260811 05:16:48.785070 1008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:48.769131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.129:55407)
0811 05:16:48.769702 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:48.769753 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:48.770568 (+ 815us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:48.771764 (+ 1196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:48.771772 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:48.772199 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:48.772757 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:48.772774 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:48.774103 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:48.774106 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:48.774977 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:48.774987 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:48.776194 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:48.776201 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:48.776407 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:48.777139 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:48.777170 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:48.778947 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:48.781268 (+ 2321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:48.781276 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:48.781292 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:48.781633 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:48.782096 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:48.782099 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:48.782101 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:48.782315 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:48.783146 (+ 831us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:48.783153 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:48.783441 (+ 288us) client_negotiation.cc:769] Sending connection context
0811 05:16:48.783640 (+ 199us) client_negotiation.cc:240] Negotiation successful
0811 05:16:48.783984 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":152,"threads_started":1}
I20260811 05:16:48.785732 1007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:48.769119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:55407 (local address 127.7.67.190:42737)
0811 05:16:48.769521 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:48.769542 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:48.769884 (+ 342us) server_negotiation.cc:407] Connection header received
0811 05:16:48.771037 (+ 1153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:48.771048 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:48.771357 (+ 309us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:48.771698 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:48.772951 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:48.773933 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:48.775120 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:48.775828 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:48.775999 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:48.779157 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:48.779163 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:48.779177 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:48.779206 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:48.781143 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:48.781787 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:48.781796 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:48.781804 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:48.781971 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:48.782433 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:48.782439 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:48.782444 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:48.783007 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:48.783159 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:48.783827 (+ 668us) server_negotiation.cc:299] Negotiation successful
0811 05:16:48.784275 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":139,"threads_started":1}
I20260811 05:16:48.785933 834 ts_tablet_manager.cc:1397] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754: Time spent bootstrapping tablet: real 0.032s user 0.030s sys 0.000s
I20260811 05:16:48.788326 1002 heartbeater.cc:347] Connected to a master server at 127.7.67.190:42737
I20260811 05:16:48.788654 1002 heartbeater.cc:464] Registering TS with master...
I20260811 05:16:48.788766 834 raft_consensus.cc:348] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } }
I20260811 05:16:48.789208 834 raft_consensus.cc:729] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16a40bffec56478e9afdaf0b92114754, State: Initialized, Role: FOLLOWER
I20260811 05:16:48.789330 1002 heartbeater.cc:511] Master 127.7.67.190:42737 requested a full tablet report, sending...
I20260811 05:16:48.789721 834 consensus_queue.cc:260] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } }
I20260811 05:16:48.790706 834 ts_tablet_manager.cc:1428] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 05:16:48.792604 717 ts_manager.cc:194] Registered new tserver with Master: 16a40bffec56478e9afdaf0b92114754 (127.7.67.129:34045)
I20260811 05:16:48.796080 717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.129@KRBTEST.COM'} at 127.7.67.129:55407
I20260811 05:16:48.799013 1002 heartbeater.cc:503] Master 127.7.67.190:42737 was elected leader, sending a full tablet report...
Aug 11 05:16:48 dist-test-slave-04bb krb5kdc[32287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425408, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for kudu/127.7.67.129@KRBTEST.COM
I20260811 05:16:48.806668 1018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:48.795603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33872 (local address 127.7.67.129:34045)
0811 05:16:48.795968 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:48.795980 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:48.796404 (+ 424us) server_negotiation.cc:407] Connection header received
0811 05:16:48.796618 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:48.796623 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:48.796836 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:48.796969 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:48.797683 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:48.798397 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:48.799418 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:48.799973 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:48.800085 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:48.802485 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:48.802495 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:48.802507 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:48.802539 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:48.804492 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:48.805011 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:48.805019 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:48.805020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:48.805124 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:48.805533 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:48.805536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:48.805538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:48.805852 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:48.805987 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:48.806321 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 05:16:48.806501 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":106,"threads_started":1}
I20260811 05:16:48.806666 1016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:48.795508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:34045 (local address 127.0.0.1:33872)
0811 05:16:48.796252 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:48.796297 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:48.796441 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:48.797045 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:48.797052 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:48.797438 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:48.797546 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:48.797554 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:48.798519 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:48.798524 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:48.799291 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:48.799301 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:48.800104 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:48.800108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:48.800235 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:48.800937 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:48.800949 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:48.802342 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:48.804638 (+ 2296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:48.804646 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:48.804651 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:48.804884 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:48.805328 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:48.805331 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:48.805332 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:48.805423 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:48.805975 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:48.805980 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:48.806091 (+ 111us) client_negotiation.cc:769] Sending connection context
0811 05:16:48.806211 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 05:16:48.806389 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":125,"threads_started":1}
I20260811 05:16:48.807978 905 tablet_service.cc:1514] Processing DeleteTablet for tablet 77c8fdbed0c24a09bc57627e6e996528 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:16:28 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:33872
I20260811 05:16:48.809018 1019 tablet_replica.cc:330] stopping tablet replica
I20260811 05:16:48.809335 1019 raft_consensus.cc:2229] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:16:48.809612 1019 raft_consensus.cc:2258] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:16:48.810629 1019 ts_tablet_manager.cc:1905] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:16:48.814329 1019 ts_tablet_manager.cc:1918] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:16:48.814447 1019 log.cc:1196] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/wal/wals/77c8fdbed0c24a09bc57627e6e996528
I20260811 05:16:48.814735 1019 ts_tablet_manager.cc:1939] T 77c8fdbed0c24a09bc57627e6e996528 P 16a40bffec56478e9afdaf0b92114754: Deleting consensus metadata
I20260811 05:16:48.816461 700 catalog_manager.cc:4841] TS 16a40bffec56478e9afdaf0b92114754 (127.7.67.129:34045): tablet 77c8fdbed0c24a09bc57627e6e996528 (table test-table [id=652ec223cc9f495cbb644bd4d5c3bc74]) successfully deleted
W20260811 05:16:48.902499 1011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:48.902769 1011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:48.902812 1011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:48.907224 1011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:48.907297 1011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:48.907358 1011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:16:48.912665 1011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:38701
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=34309
--tserver_master_addrs=127.7.67.190:42737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:16:48.913692 1011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:48.914688 1011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:48.924820 1025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:48.924837 1029 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:48.924852 1034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:48.924892 1011 server_base.cc:1034] running on GCE node
I20260811 05:16:48.925381 1011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:48.926033 1011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:48.927225 1011 hybrid_clock.cc:648] HybridClock initialized: now 1786425408927186 us; error 52 us; skew 500 ppm
Aug 11 05:16:48 dist-test-slave-04bb krb5kdc[32287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425408, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:48.930048 1011 init.cc:377] Logged in from keytab as kudu/127.7.67.130@KRBTEST.COM (short username kudu)
I20260811 05:16:48.930923 1011 webserver.cc:460] Webserver started at http://127.7.67.130:34309/ using document root <none> and password file <none>
I20260811 05:16:48.931509 1011 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:48.931623 1011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:48.935664 1011 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 05:16:48.938194 1045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:48.939589 1011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 05:16:48.939759 1011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "3ec2f199c1844981a74ee202ad6d2195"
format_stamp: "Formatted at 2026-08-11 05:16:27 on dist-test-slave-04bb"
server_key: "038e26bd0d9f3e70ab8f08d2d0b09a4f"
server_key_iv: "a4ac55458dfdc7c26b2f86c844e2bf65"
server_key_version: "encryptionkey@0"
I20260811 05:16:48.940333 1011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:48.961266 1011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:48.964422 1011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:48.964672 1011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:48.965317 1011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:48.966192 1011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:16:48.966264 1011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:48.966341 1011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:16:48.966383 1011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:48.975368 1011 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:38701
I20260811 05:16:48.975409 1213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:38701 every 8 connection(s)
I20260811 05:16:48.976387 1011 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:16:48.976828 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 1011
I20260811 05:16:48.977358 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:39339
--local_ip_for_outbound_sockets=127.7.67.131
--tserver_master_addrs=127.7.67.190:42737
--webserver_port=43077
--webserver_interface=127.7.67.131
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 05:16:48 dist-test-slave-04bb krb5kdc[32287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425408, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:16:48.993625 1007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:48.978525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:44739 (local address 127.7.67.190:42737)
0811 05:16:48.978700 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:48.978705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:48.980110 (+ 1405us) server_negotiation.cc:407] Connection header received
0811 05:16:48.981518 (+ 1408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:48.981521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:48.981579 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:48.981661 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:48.983425 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:48.983973 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:48.985930 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:48.986198 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:48.986313 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:48.989722 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:48.989726 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:48.989728 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:48.989755 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:48.991352 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:48.991957 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:48.991959 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:48.991960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:48.992037 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:48.992419 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:48.992421 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:48.992422 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:48.992669 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:48.992749 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:48.993379 (+ 630us) server_negotiation.cc:299] Negotiation successful
0811 05:16:48.993498 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 05:16:48.994680 1223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:48.978948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.130:44739)
0811 05:16:48.979907 (+ 959us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:48.979972 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:48.981225 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:48.981884 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:48.981896 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:48.982453 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:48.983232 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:48.983257 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:48.984116 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:48.984123 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:48.985767 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:48.985784 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:48.986382 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:48.986389 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:48.986631 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:48.987350 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:48.987378 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:48.989502 (+ 2124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:48.991477 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:48.991484 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:48.991500 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:48.991852 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:48.992124 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:48.992128 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:48.992130 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:48.992319 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:48.992768 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:48.992775 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:48.993091 (+ 316us) client_negotiation.cc:769] Sending connection context
0811 05:16:48.993308 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 05:16:48.993644 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"thread_start_us":540,"threads_started":1}
I20260811 05:16:48.996017 1218 heartbeater.cc:347] Connected to a master server at 127.7.67.190:42737
I20260811 05:16:48.996315 1218 heartbeater.cc:464] Registering TS with master...
I20260811 05:16:48.996935 1218 heartbeater.cc:511] Master 127.7.67.190:42737 requested a full tablet report, sending...
I20260811 05:16:48.998158 717 ts_manager.cc:194] Registered new tserver with Master: 3ec2f199c1844981a74ee202ad6d2195 (127.7.67.130:38701)
I20260811 05:16:48.998931 717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.130@KRBTEST.COM'} at 127.7.67.130:44739
W20260811 05:16:49.101413 1221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:16:49.101681 1221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:16:49.101727 1221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:16:49.106120 1221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:16:49.106209 1221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:16:49.106297 1221 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:16:49.111850 1221 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:39339
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=43077
--tserver_master_addrs=127.7.67.190:42737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:16:49.113022 1221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:16:49.114070 1221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:16:49.124392 1236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:49.124365 1233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:16:49.124388 1234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:16:49.124897 1221 server_base.cc:1034] running on GCE node
I20260811 05:16:49.125308 1221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:16:49.125967 1221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:16:49.127136 1221 hybrid_clock.cc:648] HybridClock initialized: now 1786425409127114 us; error 32 us; skew 500 ppm
Aug 11 05:16:49 dist-test-slave-04bb krb5kdc[32287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425409, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:16:49.129962 1221 init.cc:377] Logged in from keytab as kudu/127.7.67.131@KRBTEST.COM (short username kudu)
I20260811 05:16:49.130873 1221 webserver.cc:460] Webserver started at http://127.7.67.131:43077/ using document root <none> and password file <none>
I20260811 05:16:49.131479 1221 fs_manager.cc:359] Metadata directory not provided
I20260811 05:16:49.131588 1221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:16:49.136066 1221 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:16:49.138382 1247 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:49.139559 1221 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:16:49.139706 1221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "5dbd9f0248d848c5933483a5516ed43c"
format_stamp: "Formatted at 2026-08-11 05:16:28 on dist-test-slave-04bb"
server_key: "5db8d961ab4b40408ab8bdbf23fdefc3"
server_key_iv: "24b98f35ecae6c11c1a2f3e584947c2a"
server_key_version: "encryptionkey@0"
I20260811 05:16:49.140264 1221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:16:49.156064 1221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:16:49.159018 1221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:16:49.159228 1221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:16:49.159850 1221 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:16:49.160786 1221 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:16:49.160858 1221 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:49.160934 1221 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:16:49.160976 1221 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:16:49.170607 1221 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:39339
I20260811 05:16:49.170625 1427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:39339 every 8 connection(s)
I20260811 05:16:49.171661 1221 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:16:49.176419 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 1221
Aug 11 05:16:49 dist-test-slave-04bb krb5kdc[32287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425409, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:16:49.186156 1434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:49.177129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50348 (local address 127.7.67.190:42737)
0811 05:16:49.177400 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:49.177404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:49.177529 (+ 125us) server_negotiation.cc:407] Connection header received
0811 05:16:49.177710 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:49.177712 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:49.177755 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:49.177821 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:49.178714 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.179229 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.180359 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.180652 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.180755 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:49.182069 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:49.182073 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:49.182076 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:49.182101 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:49.184274 (+ 2173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:49.184797 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:49.184800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:49.184801 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:49.184875 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:49.185185 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:49.185191 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:49.185193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:49.185424 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:49.185593 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:49.185827 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 05:16:49.185945 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"spinlock_wait_cycles":768,"thread_start_us":92,"threads_started":1}
I20260811 05:16:49.186914 1007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:49.173638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:53203 (local address 127.7.67.190:42737)
0811 05:16:49.173799 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:49.173803 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:49.174753 (+ 950us) server_negotiation.cc:407] Connection header received
0811 05:16:49.176060 (+ 1307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:49.176063 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:49.176141 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:49.176245 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:49.177676 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.178533 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.179682 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.180016 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.180121 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:49.182761 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:49.182768 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:49.182773 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:49.182817 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:49.184638 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:49.185245 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:49.185248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:49.185249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:49.185316 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:49.185699 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:49.185702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:49.185703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:49.185918 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:49.186008 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:49.186657 (+ 649us) server_negotiation.cc:299] Negotiation successful
0811 05:16:49.186766 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61,"spinlock_wait_cycles":4864}
I20260811 05:16:49.187820 1431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:49.174079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.131:53203)
0811 05:16:49.174576 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:49.174621 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:49.175854 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:49.176408 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:49.176415 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:49.176901 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:16:49.177538 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:49.177554 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.178686 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.178690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:49.179563 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:49.179572 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.180161 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.180164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:49.180319 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:49.180958 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:16:49.180974 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:16:49.182589 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:16:49.184743 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:49.184751 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:49.184769 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:49.185126 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:49.185406 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:16:49.185408 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:16:49.185410 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:16:49.185590 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:16:49.186063 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:16:49.186070 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:16:49.186372 (+ 302us) client_negotiation.cc:769] Sending connection context
0811 05:16:49.186613 (+ 241us) client_negotiation.cc:240] Negotiation successful
0811 05:16:49.186939 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":115,"threads_started":1}
I20260811 05:16:49.188992 1428 heartbeater.cc:347] Connected to a master server at 127.7.67.190:42737
I20260811 05:16:49.189234 1428 heartbeater.cc:464] Registering TS with master...
I20260811 05:16:49.189723 1428 heartbeater.cc:511] Master 127.7.67.190:42737 requested a full tablet report, sending...
I20260811 05:16:49.190713 717 ts_manager.cc:194] Registered new tserver with Master: 5dbd9f0248d848c5933483a5516ed43c (127.7.67.131:39339)
I20260811 05:16:49.191251 717 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.131@KRBTEST.COM'} at 127.7.67.131:53203
I20260811 05:16:49.199863 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:16:49.203894 1007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:49.200512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50360 (local address 127.7.67.190:42737)
0811 05:16:49.200715 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:49.200721 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:49.200977 (+ 256us) server_negotiation.cc:407] Connection header received
0811 05:16:49.201108 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:49.201111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:49.201207 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:49.201300 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:16:49.201530 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.202038 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.202869 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.203068 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.203142 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:49.203436 (+ 294us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:16:49.203556 (+ 120us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 05:16:49.203755 (+ 199us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50360: token expired
Metrics: {"server-negotiator.queue_time_us":84}
W20260811 05:16:49.203966 1007 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50360: token expired
W20260811 05:16:49.204380 612 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.7.67.190:42737)
I20260811 05:16:49.211954 1007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:49.204777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50362 (local address 127.7.67.190:42737)
0811 05:16:49.204957 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:49.204962 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:49.204978 (+ 16us) server_negotiation.cc:407] Connection header received
0811 05:16:49.205096 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:49.205098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:49.205133 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:49.205194 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:16:49.205857 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.206417 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.207231 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.207428 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.207553 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:49.208996 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:16:49.208999 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:16:49.209001 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:16:49.209022 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:16:49.210419 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:49.210806 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:49.210809 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:49.210810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:49.210910 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:16:49.211195 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:16:49.211198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:16:49.211200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:16:49.211355 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:16:49.211452 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:49.211657 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 05:16:49.211753 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 05:16:49.214493 717 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50362:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:16:49.217772 717 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:16:49.226056 905 tablet_service.cc:1467] Processing CreateTablet for tablet df1d52bc5c76471fadb1dd440d8776d7 (DEFAULT_TABLE table=test-table [id=7eeb2805813d412ca4291ee85c161192]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:16:49.226794 905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df1d52bc5c76471fadb1dd440d8776d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:49.230154 834 tablet_bootstrap.cc:492] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754: Bootstrap starting.
I20260811 05:16:49.231037 1016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:49.225173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:38701 (local address 127.0.0.1:37100)
0811 05:16:49.225586 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:49.225599 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:49.225659 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:49.226803 (+ 1144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:49.226806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:49.226826 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:49.226910 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:49.226916 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.228465 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.228471 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:49.229362 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:49.229373 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.230512 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.230519 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:49.230634 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:49.230690 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 05:16:49.230773 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 05:16:49.230866 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":27,"thread_start_us":117,"threads_started":1}
I20260811 05:16:49.231160 834 tablet_bootstrap.cc:654] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:49.231202 1441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:49.225655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37100 (local address 127.7.67.130:38701)
0811 05:16:49.226017 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:49.226027 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:49.226046 (+ 19us) server_negotiation.cc:407] Connection header received
0811 05:16:49.226109 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:49.226113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:49.226291 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:49.226460 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:49.227181 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.228352 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.229493 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.230357 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.230501 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:49.230828 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:49.230887 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 05:16:49.230976 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":145,"threads_started":1}
I20260811 05:16:49.231940 1439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:49.224987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:39339 (local address 127.0.0.1:40304)
0811 05:16:49.225470 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:49.225488 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:49.225586 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:49.226470 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:49.226476 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:49.226499 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:49.226648 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:49.226657 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.228096 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.228100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:49.229415 (+ 1315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:49.229428 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.231504 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.231509 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:49.231602 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:49.231629 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 05:16:49.231710 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 05:16:49.231787 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":1}
I20260811 05:16:49.232091 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:49.225201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40304 (local address 127.7.67.131:39339)
0811 05:16:49.225929 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:49.225942 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:49.225969 (+ 27us) server_negotiation.cc:407] Connection header received
0811 05:16:49.226054 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:49.226061 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:49.226292 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:49.226460 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:49.226795 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.227973 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.229594 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.231023 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.231509 (+ 486us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:49.231680 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:49.231739 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 05:16:49.231829 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":154,"threads_started":1}
I20260811 05:16:49.234233 1115 tablet_service.cc:1467] Processing CreateTablet for tablet df1d52bc5c76471fadb1dd440d8776d7 (DEFAULT_TABLE table=test-table [id=7eeb2805813d412ca4291ee85c161192]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:16:49.234748 1319 tablet_service.cc:1467] Processing CreateTablet for tablet df1d52bc5c76471fadb1dd440d8776d7 (DEFAULT_TABLE table=test-table [id=7eeb2805813d412ca4291ee85c161192]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:16:49.235229 1115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df1d52bc5c76471fadb1dd440d8776d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:49.235847 1319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet df1d52bc5c76471fadb1dd440d8776d7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:16:49.240031 834 tablet_bootstrap.cc:492] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754: No bootstrap required, opened a new log
I20260811 05:16:49.240154 834 ts_tablet_manager.cc:1397] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260811 05:16:49.240818 834 raft_consensus.cc:348] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } }
I20260811 05:16:49.240963 834 raft_consensus.cc:374] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:49.241019 834 raft_consensus.cc:729] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16a40bffec56478e9afdaf0b92114754, State: Initialized, Role: FOLLOWER
I20260811 05:16:49.241189 834 consensus_queue.cc:260] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } }
I20260811 05:16:49.241626 834 ts_tablet_manager.cc:1428] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 05:16:49.242879 1444 tablet_bootstrap.cc:492] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195: Bootstrap starting.
I20260811 05:16:49.243693 1446 tablet_bootstrap.cc:492] T df1d52bc5c76471fadb1dd440d8776d7 P 5dbd9f0248d848c5933483a5516ed43c: Bootstrap starting.
I20260811 05:16:49.245982 1444 tablet_bootstrap.cc:654] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:49.246510 1446 tablet_bootstrap.cc:654] T df1d52bc5c76471fadb1dd440d8776d7 P 5dbd9f0248d848c5933483a5516ed43c: Neither blocks nor log segments found. Creating new log.
I20260811 05:16:49.247125 1444 log.cc:824] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:49.247289 1446 log.cc:824] T df1d52bc5c76471fadb1dd440d8776d7 P 5dbd9f0248d848c5933483a5516ed43c: Log is configured to *not* fsync() on all Append() calls
I20260811 05:16:49.248991 1444 tablet_bootstrap.cc:492] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195: No bootstrap required, opened a new log
I20260811 05:16:49.249140 1446 tablet_bootstrap.cc:492] T df1d52bc5c76471fadb1dd440d8776d7 P 5dbd9f0248d848c5933483a5516ed43c: No bootstrap required, opened a new log
I20260811 05:16:49.249214 1444 ts_tablet_manager.cc:1397] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:16:49.249356 1446 ts_tablet_manager.cc:1397] T df1d52bc5c76471fadb1dd440d8776d7 P 5dbd9f0248d848c5933483a5516ed43c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260811 05:16:49.252367 1444 raft_consensus.cc:348] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } }
I20260811 05:16:49.252689 1444 raft_consensus.cc:374] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:49.252750 1444 raft_consensus.cc:729] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ec2f199c1844981a74ee202ad6d2195, State: Initialized, Role: FOLLOWER
I20260811 05:16:49.252504 1446 raft_consensus.cc:348] T df1d52bc5c76471fadb1dd440d8776d7 P 5dbd9f0248d848c5933483a5516ed43c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } }
I20260811 05:16:49.252871 1446 raft_consensus.cc:374] T df1d52bc5c76471fadb1dd440d8776d7 P 5dbd9f0248d848c5933483a5516ed43c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:16:49.252935 1446 raft_consensus.cc:729] T df1d52bc5c76471fadb1dd440d8776d7 P 5dbd9f0248d848c5933483a5516ed43c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dbd9f0248d848c5933483a5516ed43c, State: Initialized, Role: FOLLOWER
I20260811 05:16:49.253187 1444 consensus_queue.cc:260] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } }
I20260811 05:16:49.253449 1446 consensus_queue.cc:260] T df1d52bc5c76471fadb1dd440d8776d7 P 5dbd9f0248d848c5933483a5516ed43c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } }
I20260811 05:16:49.253860 1218 heartbeater.cc:503] Master 127.7.67.190:42737 was elected leader, sending a full tablet report...
I20260811 05:16:49.254072 1428 heartbeater.cc:503] Master 127.7.67.190:42737 was elected leader, sending a full tablet report...
I20260811 05:16:49.254252 1444 ts_tablet_manager.cc:1428] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 05:16:49.254470 1446 ts_tablet_manager.cc:1428] T df1d52bc5c76471fadb1dd440d8776d7 P 5dbd9f0248d848c5933483a5516ed43c: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
W20260811 05:16:49.268177 1006 tablet.cc:2369] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:16:49.311573 1452 raft_consensus.cc:482] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:16:49.311782 1452 raft_consensus.cc:504] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } }
I20260811 05:16:49.313035 1452 leader_election.cc:288] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5dbd9f0248d848c5933483a5516ed43c (127.7.67.131:39339), 16a40bffec56478e9afdaf0b92114754 (127.7.67.129:34045)
I20260811 05:16:49.317265 1223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:49.313337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:39339 (local address 127.7.67.130:38483)
0811 05:16:49.313502 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:49.313520 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:49.313631 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:49.313984 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:49.313988 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:49.314022 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:49.314168 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:49.314177 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.315044 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.315048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:49.316015 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:49.316026 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.316784 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.316788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:49.316902 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:49.316930 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 05:16:49.317026 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 05:16:49.317114 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 05:16:49.317277 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:49.313469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:38483 (local address 127.7.67.131:39339)
0811 05:16:49.313600 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:49.313604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:49.313619 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:16:49.313778 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:49.313782 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:49.313841 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:49.313932 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:49.314313 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.314920 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.316155 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.316660 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.316763 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:49.316806 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:49.317042 (+ 236us) server_negotiation.cc:299] Negotiation successful
0811 05:16:49.317120 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:16:49.317622 1456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:49.313337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:34045 (local address 127.7.67.130:42779)
0811 05:16:49.313844 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:49.313857 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:16:49.313953 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:16:49.314311 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:16:49.314314 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:16:49.314332 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:16:49.314447 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:49.314453 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.315355 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.315357 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:49.316181 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:16:49.316192 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.317164 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.317167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:16:49.317265 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:49.317295 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 05:16:49.317369 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 05:16:49.317444 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":143,"thread_start_us":119,"threads_started":1}
I20260811 05:16:49.317616 1455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:49.313477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:42779 (local address 127.7.67.129:34045)
0811 05:16:49.313808 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:49.313811 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:49.313944 (+ 133us) server_negotiation.cc:407] Connection header received
0811 05:16:49.314055 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:49.314058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:49.314131 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:49.314284 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:16:49.314562 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.315229 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.316328 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.317042 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.317157 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:49.317352 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:49.317398 (+ 46us) server_negotiation.cc:299] Negotiation successful
0811 05:16:49.317486 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":111,"threads_started":1}
I20260811 05:16:49.318038 1352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df1d52bc5c76471fadb1dd440d8776d7" candidate_uuid: "3ec2f199c1844981a74ee202ad6d2195" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dbd9f0248d848c5933483a5516ed43c" is_pre_election: true
I20260811 05:16:49.318306 935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df1d52bc5c76471fadb1dd440d8776d7" candidate_uuid: "3ec2f199c1844981a74ee202ad6d2195" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16a40bffec56478e9afdaf0b92114754" is_pre_election: true
I20260811 05:16:49.318364 1352 raft_consensus.cc:2454] T df1d52bc5c76471fadb1dd440d8776d7 P 5dbd9f0248d848c5933483a5516ed43c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ec2f199c1844981a74ee202ad6d2195 in term 0.
I20260811 05:16:49.318596 935 raft_consensus.cc:2454] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ec2f199c1844981a74ee202ad6d2195 in term 0.
I20260811 05:16:49.318893 1046 leader_election.cc:302] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ec2f199c1844981a74ee202ad6d2195, 5dbd9f0248d848c5933483a5516ed43c; no voters:
I20260811 05:16:49.319222 1452 raft_consensus.cc:2781] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:16:49.319331 1452 raft_consensus.cc:482] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:16:49.319387 1452 raft_consensus.cc:3037] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:49.320499 1452 raft_consensus.cc:504] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } }
I20260811 05:16:49.320922 1452 leader_election.cc:288] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195 [CANDIDATE]: Term 1 election: Requested vote from peers 5dbd9f0248d848c5933483a5516ed43c (127.7.67.131:39339), 16a40bffec56478e9afdaf0b92114754 (127.7.67.129:34045)
I20260811 05:16:49.321328 935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df1d52bc5c76471fadb1dd440d8776d7" candidate_uuid: "3ec2f199c1844981a74ee202ad6d2195" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "16a40bffec56478e9afdaf0b92114754"
I20260811 05:16:49.321336 1352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "df1d52bc5c76471fadb1dd440d8776d7" candidate_uuid: "3ec2f199c1844981a74ee202ad6d2195" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dbd9f0248d848c5933483a5516ed43c"
I20260811 05:16:49.321439 935 raft_consensus.cc:3037] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:49.321461 1352 raft_consensus.cc:3037] T df1d52bc5c76471fadb1dd440d8776d7 P 5dbd9f0248d848c5933483a5516ed43c [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:16:49.322450 935 raft_consensus.cc:2454] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ec2f199c1844981a74ee202ad6d2195 in term 1.
I20260811 05:16:49.322540 1352 raft_consensus.cc:2454] T df1d52bc5c76471fadb1dd440d8776d7 P 5dbd9f0248d848c5933483a5516ed43c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ec2f199c1844981a74ee202ad6d2195 in term 1.
I20260811 05:16:49.322870 1046 leader_election.cc:302] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ec2f199c1844981a74ee202ad6d2195, 5dbd9f0248d848c5933483a5516ed43c; no voters:
I20260811 05:16:49.323086 1452 raft_consensus.cc:2781] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:16:49.323328 1452 raft_consensus.cc:686] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195 [term 1 LEADER]: Becoming Leader. State: Replica: 3ec2f199c1844981a74ee202ad6d2195, State: Running, Role: LEADER
I20260811 05:16:49.323681 1452 consensus_queue.cc:237] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } }
I20260811 05:16:49.327060 717 catalog_manager.cc:5495] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ec2f199c1844981a74ee202ad6d2195 (127.7.67.130). New cstate: current_term: 1 leader_uuid: "3ec2f199c1844981a74ee202ad6d2195" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } health_report { overall_health: UNKNOWN } } }
I20260811 05:16:49.358968 1441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:49.354682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37104 (local address 127.7.67.130:38701)
0811 05:16:49.354868 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 05:16:49.354875 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:16:49.354997 (+ 122us) server_negotiation.cc:407] Connection header received
0811 05:16:49.355244 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:16:49.355247 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:16:49.355312 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:16:49.355460 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:16:49.355741 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.356309 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.357311 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:16:49.357556 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:16:49.357682 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:16:49.358040 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:16:49.358488 (+ 448us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:16:49.358636 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 05:16:49.358723 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 05:16:49.358783 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 05:16:49.365351 1352 raft_consensus.cc:1261] T df1d52bc5c76471fadb1dd440d8776d7 P 5dbd9f0248d848c5933483a5516ed43c [term 1 FOLLOWER]: Refusing update from remote peer 3ec2f199c1844981a74ee202ad6d2195: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:16:49.365411 935 raft_consensus.cc:1261] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Refusing update from remote peer 3ec2f199c1844981a74ee202ad6d2195: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:16:49.366092 1452 consensus_queue.cc:1035] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:16:49.366331 1457 consensus_queue.cc:1035] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195 [LEADER]: Connected to new peer: Peer: permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:16:49.373369 1462 mvcc.cc:205] Tried to move back new op lower bound from 7317198476752773120 to 7317198476592312320. Current Snapshot: MvccSnapshot[applied={T|T < 7317198476752773120}]
I20260811 05:16:49.378392 1463 mvcc.cc:205] Tried to move back new op lower bound from 7317198476752773120 to 7317198476592312320. Current Snapshot: MvccSnapshot[applied={T|T < 7317198476752773120}]
I20260811 05:16:49.381776 716 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:16:49.381919 716 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50362:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:16:49.384348 716 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94: Sending DeleteTablet for 3 replicas of tablet df1d52bc5c76471fadb1dd440d8776d7
I20260811 05:16:49.385139 1115 tablet_service.cc:1514] Processing DeleteTablet for tablet df1d52bc5c76471fadb1dd440d8776d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:16:49 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:37100
I20260811 05:16:49.385422 905 tablet_service.cc:1514] Processing DeleteTablet for tablet df1d52bc5c76471fadb1dd440d8776d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:16:49 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:33872
I20260811 05:16:49.385677 1319 tablet_service.cc:1514] Processing DeleteTablet for tablet df1d52bc5c76471fadb1dd440d8776d7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:16:49 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:40304
I20260811 05:16:49.385761 1478 tablet_replica.cc:330] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195: stopping tablet replica
I20260811 05:16:49.385905 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 772
I20260811 05:16:49.386234 1478 raft_consensus.cc:2229] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:16:49.386603 1480 tablet_replica.cc:330] T df1d52bc5c76471fadb1dd440d8776d7 P 5dbd9f0248d848c5933483a5516ed43c: stopping tablet replica
I20260811 05:16:49.386858 1480 raft_consensus.cc:2229] T df1d52bc5c76471fadb1dd440d8776d7 P 5dbd9f0248d848c5933483a5516ed43c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:16:49.387109 1480 raft_consensus.cc:2258] T df1d52bc5c76471fadb1dd440d8776d7 P 5dbd9f0248d848c5933483a5516ed43c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:16:49.398064 1478 raft_consensus.cc:2258] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:16:49.400463 1480 ts_tablet_manager.cc:1905] T df1d52bc5c76471fadb1dd440d8776d7 P 5dbd9f0248d848c5933483a5516ed43c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:16:49.400664 1478 ts_tablet_manager.cc:1905] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:16:49.403412 1480 ts_tablet_manager.cc:1918] T df1d52bc5c76471fadb1dd440d8776d7 P 5dbd9f0248d848c5933483a5516ed43c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:16:49.403515 1480 log.cc:1196] T df1d52bc5c76471fadb1dd440d8776d7 P 5dbd9f0248d848c5933483a5516ed43c: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/df1d52bc5c76471fadb1dd440d8776d7
I20260811 05:16:49.403872 1480 ts_tablet_manager.cc:1939] T df1d52bc5c76471fadb1dd440d8776d7 P 5dbd9f0248d848c5933483a5516ed43c: Deleting consensus metadata
I20260811 05:16:49.404417 1478 ts_tablet_manager.cc:1918] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:16:49.404614 1478 log.cc:1196] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/df1d52bc5c76471fadb1dd440d8776d7
I20260811 05:16:49.405028 1478 ts_tablet_manager.cc:1939] T df1d52bc5c76471fadb1dd440d8776d7 P 3ec2f199c1844981a74ee202ad6d2195: Deleting consensus metadata
I20260811 05:16:49.405409 696 catalog_manager.cc:4841] TS 5dbd9f0248d848c5933483a5516ed43c (127.7.67.131:39339): tablet df1d52bc5c76471fadb1dd440d8776d7 (table test-table [id=7eeb2805813d412ca4291ee85c161192]) successfully deleted
I20260811 05:16:49.406464 701 catalog_manager.cc:4841] TS 3ec2f199c1844981a74ee202ad6d2195 (127.7.67.130:38701): tablet df1d52bc5c76471fadb1dd440d8776d7 (table test-table [id=7eeb2805813d412ca4291ee85c161192]) successfully deleted
W20260811 05:16:49.408740 700 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.129:34045 (error 108)
I20260811 05:16:49.409858 1439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:16:49.409534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:34045 (local address 127.0.0.1:33882)
0811 05:16:49.409667 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 05:16:49.409754 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.129:34045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 05:16:49.410036 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 1011
W20260811 05:16:49.416752 700 catalog_manager.cc:4584] TS 16a40bffec56478e9afdaf0b92114754 (127.7.67.129:34045): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet df1d52bc5c76471fadb1dd440d8776d7: Network error: Client connection negotiation failed: client connection to 127.7.67.129:34045: connect: Connection refused (error 111)
I20260811 05:16:49.424636 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 1221
I20260811 05:16:49.442173 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 664
I20260811 05:17:09.457275 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:42737
--webserver_interface=127.7.67.190
--webserver_port=35313
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:42737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:09.577700 1482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:09.577972 1482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:09.578013 1482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:09.582046 1482 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 05:17:09.582098 1482 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 05:17:09.582114 1482 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 05:17:09.582129 1482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:09.582147 1482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:09.582170 1482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:09.582187 1482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:09.582203 1482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:09.587508 1482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:42737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:42737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=35313
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:09.588651 1482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:09.589685 1482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:17:09.599033 1482 server_base.cc:1034] running on GCE node
W20260811 05:17:09.599025 1495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:09.598984 1492 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:09.598984 1491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:09.600219 1482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:09.601424 1482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:09.602622 1482 hybrid_clock.cc:648] HybridClock initialized: now 1786425429602604 us; error 32 us; skew 500 ppm
Aug 11 05:17:09 dist-test-slave-04bb krb5kdc[32287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425429, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:09.610052 1482 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:17:09.611047 1482 webserver.cc:460] Webserver started at http://127.7.67.190:35313/ using document root <none> and password file <none>
I20260811 05:17:09.611665 1482 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:09.611765 1482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:09.615844 1482 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 05:17:09.618253 1509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:09.619729 1482 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260811 05:17:09.619886 1482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "ffae156278c1481c9d8ecebae8f56f94"
format_stamp: "Formatted at 2026-08-11 05:16:27 on dist-test-slave-04bb"
server_key: "08f8e89093750e0a825ee8e46987e228"
server_key_iv: "67368a5d2120912ad7fab1e92fa16432"
server_key_version: "encryptionkey@0"
I20260811 05:17:09.620484 1482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:09.636303 1482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:09.639626 1482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:09.639848 1482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:09.647492 1482 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:42737
I20260811 05:17:09.647815 1583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:42737 every 8 connection(s)
I20260811 05:17:09.648808 1482 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:09.653467 1584 sys_catalog.cc:263] Verifying existing consensus state
I20260811 05:17:09.655272 1584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94: Bootstrap starting.
I20260811 05:17:09.655963 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 1482
I20260811 05:17:09.656478 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:34045
--local_ip_for_outbound_sockets=127.7.67.129
--tserver_master_addrs=127.7.67.190:42737
--webserver_port=43991
--webserver_interface=127.7.67.129
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:17:09.673672 1584 log.cc:824] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:09.685317 1584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:17:09.685724 1584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94: Bootstrap complete.
I20260811 05:17:09.689918 1584 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffae156278c1481c9d8ecebae8f56f94" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42737 } }
I20260811 05:17:09.690330 1584 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffae156278c1481c9d8ecebae8f56f94, State: Initialized, Role: FOLLOWER
I20260811 05:17:09.690888 1584 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffae156278c1481c9d8ecebae8f56f94" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42737 } }
I20260811 05:17:09.691042 1584 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:09.691135 1584 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:09.691257 1584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 2 FOLLOWER]: Advancing to term 3
I20260811 05:17:09.693205 1584 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffae156278c1481c9d8ecebae8f56f94" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42737 } }
I20260811 05:17:09.693555 1584 leader_election.cc:302] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffae156278c1481c9d8ecebae8f56f94; no voters:
I20260811 05:17:09.693881 1584 leader_election.cc:288] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 05:17:09.694018 1591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 05:17:09.694321 1591 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [term 3 LEADER]: Becoming Leader. State: Replica: ffae156278c1481c9d8ecebae8f56f94, State: Running, Role: LEADER
I20260811 05:17:09.694677 1591 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffae156278c1481c9d8ecebae8f56f94" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42737 } }
I20260811 05:17:09.695088 1584 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:09.695649 1591 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffae156278c1481c9d8ecebae8f56f94. Latest consensus state: current_term: 3 leader_uuid: "ffae156278c1481c9d8ecebae8f56f94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffae156278c1481c9d8ecebae8f56f94" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42737 } } }
I20260811 05:17:09.695768 1591 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:09.696420 1592 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ffae156278c1481c9d8ecebae8f56f94" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffae156278c1481c9d8ecebae8f56f94" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 42737 } } }
I20260811 05:17:09.696517 1592 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:09.697219 1595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:09.702260 1595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:09.703015 1595 catalog_manager.cc:1259] Loaded cluster ID: bd06e0924a30480ba9d1f9040f938480
I20260811 05:17:09.703083 1595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:09.705651 1595 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:09.706439 1595 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94: Loaded TSK: 0
I20260811 05:17:09.706528 1595 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ffae156278c1481c9d8ecebae8f56f94: Loaded TSK: 1
F20260811 05:17:09.707305 1595 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7feecf0ce9ad google::LogMessage::Fail() at ??:0
@ 0x7feecf0d188a google::LogMessage::SendToLog() at ??:0
@ 0x7feecf0ce4de google::LogMessage::Flush() at ??:0
@ 0x7feecf0cfa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7feed1d9218d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7feed1d8af0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7feed9a337bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7feed9a0d99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7feed9a0e178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7feed9a4fdf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7feed9a6e82b std::function<>::operator()() at ??:0
@ 0x7feed9a0db4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7feed9a0ef3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7feed9a0b579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7feed9a4f640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7feed13203d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7feed1320c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7feed13223f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7feed1311512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7feed29556db start_thread at ??:0
@ 0x7feecdbcc61f clone at ??:0
W20260811 05:17:09.810406 1588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:09.810806 1588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:09.810879 1588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:09.817183 1588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:09.817276 1588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:09.817365 1588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:17:09.825735 1588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:34045
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=43991
--tserver_master_addrs=127.7.67.190:42737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:09.827198 1588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:09.828689 1588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:09.840308 1657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:09.840324 1661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:09.840309 1655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:09.840939 1588 server_base.cc:1034] running on GCE node
I20260811 05:17:09.841396 1588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:09.842207 1588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:09.843411 1588 hybrid_clock.cc:648] HybridClock initialized: now 1786425429843388 us; error 35 us; skew 500 ppm
Aug 11 05:17:09 dist-test-slave-04bb krb5kdc[32287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425429, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:09.846912 1588 init.cc:377] Logged in from keytab as kudu/127.7.67.129@KRBTEST.COM (short username kudu)
I20260811 05:17:09.847828 1588 webserver.cc:460] Webserver started at http://127.7.67.129:43991/ using document root <none> and password file <none>
I20260811 05:17:09.848457 1588 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:09.848568 1588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:09.852768 1588 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260811 05:17:09.855206 1671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:09.856784 1588 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:17:09.856930 1588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "16a40bffec56478e9afdaf0b92114754"
format_stamp: "Formatted at 2026-08-11 05:16:27 on dist-test-slave-04bb"
server_key: "e466aba3d0cc6274a6f0526862a2242c"
server_key_iv: "da64a65071ae364bcf9ea510c8655afe"
server_key_version: "encryptionkey@0"
I20260811 05:17:09.857575 1588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:09.878630 1588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:09.882372 1588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:09.882602 1588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:09.883293 1588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:09.884773 1679 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 05:17:09.890480 1588 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 05:17:09.890597 1588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.001s
I20260811 05:17:09.890699 1588 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 05:17:09.892683 1588 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 05:17:09.892755 1588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.002s
I20260811 05:17:09.893081 1679 tablet_bootstrap.cc:492] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754: Bootstrap starting.
I20260811 05:17:09.905004 1588 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:34045
I20260811 05:17:09.905025 1837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:34045 every 8 connection(s)
I20260811 05:17:09.906252 1588 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:17:09.915185 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 1588
I20260811 05:17:09.915207 1679 log.cc:824] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:09.915936 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:38701
--local_ip_for_outbound_sockets=127.7.67.130
--tserver_master_addrs=127.7.67.190:42737
--webserver_port=34309
--webserver_interface=127.7.67.130
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:17:09.924463 1679 tablet_bootstrap.cc:492] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 05:17:09.924954 1679 tablet_bootstrap.cc:492] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754: Bootstrap complete.
I20260811 05:17:09.926426 1679 ts_tablet_manager.cc:1397] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754: Time spent bootstrapping tablet: real 0.034s user 0.012s sys 0.019s
I20260811 05:17:09.929826 1679 raft_consensus.cc:348] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } }
I20260811 05:17:09.930383 1679 raft_consensus.cc:729] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16a40bffec56478e9afdaf0b92114754, State: Initialized, Role: FOLLOWER
I20260811 05:17:09.930932 1679 consensus_queue.cc:260] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } }
I20260811 05:17:09.932053 1679 ts_tablet_manager.cc:1428] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
W20260811 05:17:09.970017 1842 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:17:09.908441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.129:59903)
0811 05:17:09.909068 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:09.909129 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:09.910036 (+ 907us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:09.969174 (+ 59138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":161,"threads_started":1}
W20260811 05:17:09.970602 1838 heartbeater.cc:644] Failed to heartbeat to 127.7.67.190:42737 (0 consecutive failures): Network error: Failed to ping master at 127.7.67.190:42737: Client connection negotiation failed: client connection to 127.7.67.190:42737: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 05:17:09.971395 1842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:09.971132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.129:49173)
0811 05:17:09.971271 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:09.971337 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 05:17:09.972111 1842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:09.971899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.129:55215)
0811 05:17:09.972010 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:09.972061 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 05:17:09.972316 1838 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 05:17:10.040979 1845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:10.041225 1845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:10.041299 1845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:10.045401 1845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:10.045457 1845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:10.045543 1845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:17:10.050750 1845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:38701
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=34309
--tserver_master_addrs=127.7.67.190:42737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:10.051800 1845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:10.052898 1845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:10.062734 1856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:10.062752 1855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:10.062734 1861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:10.063220 1845 server_base.cc:1034] running on GCE node
I20260811 05:17:10.063637 1845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:10.064270 1845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:10.065436 1845 hybrid_clock.cc:648] HybridClock initialized: now 1786425430065405 us; error 42 us; skew 500 ppm
Aug 11 05:17:10 dist-test-slave-04bb krb5kdc[32287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425430, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:10.068418 1845 init.cc:377] Logged in from keytab as kudu/127.7.67.130@KRBTEST.COM (short username kudu)
I20260811 05:17:10.069411 1845 webserver.cc:460] Webserver started at http://127.7.67.130:34309/ using document root <none> and password file <none>
I20260811 05:17:10.069931 1845 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:10.069990 1845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:10.074048 1845 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:17:10.076239 1868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:10.077575 1845 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:17:10.077679 1845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "3ec2f199c1844981a74ee202ad6d2195"
format_stamp: "Formatted at 2026-08-11 05:16:27 on dist-test-slave-04bb"
server_key: "038e26bd0d9f3e70ab8f08d2d0b09a4f"
server_key_iv: "a4ac55458dfdc7c26b2f86c844e2bf65"
server_key_version: "encryptionkey@0"
I20260811 05:17:10.078218 1845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:10.093607 1845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:10.098711 1845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:10.098922 1845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:10.099711 1845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:10.100876 1845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:10.100935 1845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:10.101001 1845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:10.101033 1845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:10.110519 1845 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:38701
I20260811 05:17:10.110548 2050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:38701 every 8 connection(s)
I20260811 05:17:10.111553 1845 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:17:10.114342 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 1845
I20260811 05:17:10.114805 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:39339
--local_ip_for_outbound_sockets=127.7.67.131
--tserver_master_addrs=127.7.67.190:42737
--webserver_port=43077
--webserver_interface=127.7.67.131
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:17:10.115609 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:10.113691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.130:45991)
0811 05:17:10.114160 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:10.114277 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":109,"threads_started":1}
W20260811 05:17:10.116094 2051 heartbeater.cc:644] Failed to heartbeat to 127.7.67.190:42737 (0 consecutive failures): Network error: Failed to ping master at 127.7.67.190:42737: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
I20260811 05:17:10.116835 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:10.116544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.130:46791)
0811 05:17:10.116691 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:10.116756 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 05:17:10.117544 2054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:10.117286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.130:50393)
0811 05:17:10.117415 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:10.117476 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 05:17:10.117726 2051 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 05:17:10.234508 2055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:10.234808 2055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:10.234918 2055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:10.239123 2055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:10.239205 2055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:10.239326 2055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:17:10.244652 2055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:39339
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=43077
--tserver_master_addrs=127.7.67.190:42737
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:10.245744 2055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:10.246790 2055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:17:10.256959 2055 server_base.cc:1034] running on GCE node
W20260811 05:17:10.256911 2062 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:10.256911 2064 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:10.256938 2061 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:10.259186 2055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:10.259840 2055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:10.261003 2055 hybrid_clock.cc:648] HybridClock initialized: now 1786425430260967 us; error 49 us; skew 500 ppm
Aug 11 05:17:10 dist-test-slave-04bb krb5kdc[32287](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425430, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:10.264176 2055 init.cc:377] Logged in from keytab as kudu/127.7.67.131@KRBTEST.COM (short username kudu)
I20260811 05:17:10.265327 2055 webserver.cc:460] Webserver started at http://127.7.67.131:43077/ using document root <none> and password file <none>
I20260811 05:17:10.265923 2055 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:10.266024 2055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:10.270129 2055 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:17:10.272234 2077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:10.273587 2055 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:17:10.273756 2055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "5dbd9f0248d848c5933483a5516ed43c"
format_stamp: "Formatted at 2026-08-11 05:16:28 on dist-test-slave-04bb"
server_key: "5db8d961ab4b40408ab8bdbf23fdefc3"
server_key_iv: "24b98f35ecae6c11c1a2f3e584947c2a"
server_key_version: "encryptionkey@0"
I20260811 05:17:10.274290 2055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:10.297963 2055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:10.301398 2055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:10.301579 2055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:10.302129 2055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:10.302928 2055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:10.302975 2055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:10.303017 2055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:10.303032 2055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:10.312086 2055 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:39339
I20260811 05:17:10.312114 2257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:39339 every 8 connection(s)
I20260811 05:17:10.313143 2055 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:17:10.316998 2267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:10.315232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.131:50817)
0811 05:17:10.316144 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:10.316260 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":714,"thread_start_us":550,"threads_started":1}
W20260811 05:17:10.317529 2262 heartbeater.cc:644] Failed to heartbeat to 127.7.67.190:42737 (0 consecutive failures): Network error: Failed to ping master at 127.7.67.190:42737: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
I20260811 05:17:10.318069 2267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:10.317856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.131:52181)
0811 05:17:10.317967 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:10.318014 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 05:17:10.318732 2267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:10.318507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.131:44273)
0811 05:17:10.318618 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:10.318681 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 05:17:10.318940 2262 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 05:17:10.323042 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 2055
I20260811 05:17:10.973438 2271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:10.972985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.129:39359)
0811 05:17:10.973274 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:10.973334 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":80,"threads_started":1}
I20260811 05:17:11.118680 2272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:11.118295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.130:54367)
0811 05:17:11.118532 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:11.118583 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":71,"threads_started":1}
I20260811 05:17:11.320109 2274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:11.319643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.131:49649)
0811 05:17:11.319940 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:11.320002 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":88,"threads_started":1}
I20260811 05:17:11.578614 2275 raft_consensus.cc:482] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:17:11.578933 2275 raft_consensus.cc:504] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } }
I20260811 05:17:11.580240 2275 leader_election.cc:288] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5dbd9f0248d848c5933483a5516ed43c (127.7.67.131:39339), 3ec2f199c1844981a74ee202ad6d2195 (127.7.67.130:38701)
Aug 11 05:17:11 dist-test-slave-04bb krb5kdc[32287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425429, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.130@KRBTEST.COM
Aug 11 05:17:11 dist-test-slave-04bb krb5kdc[32287](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425429, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.131@KRBTEST.COM
I20260811 05:17:11.599963 2277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:11.580492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:38701 (local address 127.7.67.129:33285)
0811 05:17:11.580883 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:11.580900 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:11.581029 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:11.583251 (+ 2222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:11.583258 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:11.583908 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:11.584810 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:11.584834 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:11.586878 (+ 2044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:11.586882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:11.588582 (+ 1700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:11.588593 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:11.589653 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:11.589657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:11.589979 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:11.590961 (+ 982us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:11.590983 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:11.593134 (+ 2151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:11.596467 (+ 3333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:11.596475 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:11.596490 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:11.597058 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:11.597513 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:11.597517 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:11.597519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:11.597772 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:11.598602 (+ 830us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:11.598609 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:11.599027 (+ 418us) client_negotiation.cc:769] Sending connection context
0811 05:17:11.599259 (+ 232us) client_negotiation.cc:240] Negotiation successful
0811 05:17:11.599788 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":635,"spinlock_wait_cycles":48896,"thread_start_us":86,"threads_started":1}
I20260811 05:17:11.599967 2279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:11.580716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:33285 (local address 127.7.67.130:38701)
0811 05:17:11.581188 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:11.581210 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:11.581257 (+ 47us) server_negotiation.cc:407] Connection header received
0811 05:17:11.582267 (+ 1010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:11.582277 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:11.582761 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:11.583132 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:11.585036 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:11.586679 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:11.588717 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:11.589508 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:11.589693 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:11.593787 (+ 4094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:11.593795 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:11.593813 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:11.593848 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:11.596332 (+ 2484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:11.597205 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:11.597212 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:11.597218 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:11.597393 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:11.597897 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:11.597901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:11.597903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:11.598461 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:11.598605 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:11.599445 (+ 840us) server_negotiation.cc:299] Negotiation successful
0811 05:17:11.599774 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":90,"threads_started":1}
I20260811 05:17:11.599996 2278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:11.580716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:54823 (local address 127.7.67.131:39339)
0811 05:17:11.581166 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:11.581203 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:11.581257 (+ 54us) server_negotiation.cc:407] Connection header received
0811 05:17:11.582267 (+ 1010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:11.582277 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:11.582761 (+ 484us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:11.583131 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:11.585036 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:11.586679 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:11.588717 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:11.589507 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:11.589693 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:11.593787 (+ 4094us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:11.593795 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:11.593813 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:11.593848 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:11.596320 (+ 2472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:11.597205 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:11.597212 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:11.597218 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:11.597393 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:11.597897 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:11.597901 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:11.597903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:11.598461 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:11.598605 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:11.599445 (+ 840us) server_negotiation.cc:299] Negotiation successful
0811 05:17:11.599756 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":75,"threads_started":1}
I20260811 05:17:11.599963 2276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:11.580495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:39339 (local address 127.7.67.129:54823)
0811 05:17:11.580985 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:11.580998 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:11.581064 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:11.583251 (+ 2187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:11.583258 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:11.583908 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:11.584810 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:11.584834 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:11.586878 (+ 2044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:11.586882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:11.588581 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:11.588593 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:11.589653 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:11.589657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:11.589978 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:11.590975 (+ 997us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:11.590988 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:11.593530 (+ 2542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:11.596467 (+ 2937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:11.596475 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:11.596490 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:11.597058 (+ 568us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:11.597513 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:11.597517 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:11.597519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:11.597772 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:11.598602 (+ 830us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:11.598609 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:11.599027 (+ 418us) client_negotiation.cc:769] Sending connection context
0811 05:17:11.599259 (+ 232us) client_negotiation.cc:240] Negotiation successful
0811 05:17:11.599729 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":473,"spinlock_wait_cycles":81536,"thread_start_us":63,"threads_started":1}
W20260811 05:17:11.603160 1676 leader_election.cc:341] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ec2f199c1844981a74ee202ad6d2195 (127.7.67.130:38701): Not found: Tablet not found: df1d52bc5c76471fadb1dd440d8776d7
W20260811 05:17:11.603632 1672 leader_election.cc:341] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5dbd9f0248d848c5933483a5516ed43c (127.7.67.131:39339): Not found: Tablet not found: df1d52bc5c76471fadb1dd440d8776d7
I20260811 05:17:11.603709 1672 leader_election.cc:302] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16a40bffec56478e9afdaf0b92114754; no voters: 3ec2f199c1844981a74ee202ad6d2195, 5dbd9f0248d848c5933483a5516ed43c
I20260811 05:17:11.604028 2275 raft_consensus.cc:2726] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:17:11.974560 2276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:11.974250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.129:55259)
0811 05:17:11.974404 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:11.974471 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 05:17:12.120026 2280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:12.119583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.130:55397)
0811 05:17:12.119887 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:12.119941 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":129,"threads_started":1}
I20260811 05:17:12.321393 2282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:12.320955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.131:37791)
0811 05:17:12.321232 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:12.321299 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":79,"threads_started":1}
I20260811 05:17:12.975766 2283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:12.975302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.129:54693)
0811 05:17:12.975605 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:12.975664 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":102,"threads_started":1}
I20260811 05:17:13.121239 2284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:13.120746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.130:50371)
0811 05:17:13.121065 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:13.121128 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":125,"threads_started":1}
I20260811 05:17:13.322566 2288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:13.322169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.131:58541)
0811 05:17:13.322409 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:13.322462 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":77,"threads_started":1}
I20260811 05:17:13.766021 2289 raft_consensus.cc:482] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:17:13.766211 2289 raft_consensus.cc:504] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } }
I20260811 05:17:13.766755 2289 leader_election.cc:288] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5dbd9f0248d848c5933483a5516ed43c (127.7.67.131:39339), 3ec2f199c1844981a74ee202ad6d2195 (127.7.67.130:38701)
W20260811 05:17:13.767678 1672 leader_election.cc:341] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5dbd9f0248d848c5933483a5516ed43c (127.7.67.131:39339): Not found: Tablet not found: df1d52bc5c76471fadb1dd440d8776d7
W20260811 05:17:13.767764 1676 leader_election.cc:341] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ec2f199c1844981a74ee202ad6d2195 (127.7.67.130:38701): Not found: Tablet not found: df1d52bc5c76471fadb1dd440d8776d7
I20260811 05:17:13.767895 1676 leader_election.cc:302] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16a40bffec56478e9afdaf0b92114754; no voters: 3ec2f199c1844981a74ee202ad6d2195, 5dbd9f0248d848c5933483a5516ed43c
I20260811 05:17:13.768076 2289 raft_consensus.cc:2726] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:17:13.977082 2291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:13.976630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.129:36611)
0811 05:17:13.976921 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:13.976979 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":104,"threads_started":1}
I20260811 05:17:14.122419 2292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:14.122009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.130:58849)
0811 05:17:14.122259 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:14.122328 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":100,"threads_started":1}
I20260811 05:17:14.323683 2293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:14.323282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.131:57975)
0811 05:17:14.323530 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:14.323583 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":76,"threads_started":1}
I20260811 05:17:14.978871 2294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:14.978346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.129:49849)
0811 05:17:14.978693 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:14.978754 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":81,"threads_started":1}
I20260811 05:17:15.123750 2295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:15.123243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.130:41611)
0811 05:17:15.123569 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:15.123642 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":104,"threads_started":1}
I20260811 05:17:15.325006 2296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:15.324518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.131:43907)
0811 05:17:15.324852 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:15.324909 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":83,"threads_started":1}
I20260811 05:17:15.544026 2297 raft_consensus.cc:482] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:17:15.544217 2297 raft_consensus.cc:504] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } }
I20260811 05:17:15.544808 2297 leader_election.cc:288] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5dbd9f0248d848c5933483a5516ed43c (127.7.67.131:39339), 3ec2f199c1844981a74ee202ad6d2195 (127.7.67.130:38701)
W20260811 05:17:15.545645 1676 leader_election.cc:341] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ec2f199c1844981a74ee202ad6d2195 (127.7.67.130:38701): Not found: Tablet not found: df1d52bc5c76471fadb1dd440d8776d7
W20260811 05:17:15.545785 1672 leader_election.cc:341] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5dbd9f0248d848c5933483a5516ed43c (127.7.67.131:39339): Not found: Tablet not found: df1d52bc5c76471fadb1dd440d8776d7
I20260811 05:17:15.545881 1672 leader_election.cc:302] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16a40bffec56478e9afdaf0b92114754; no voters: 3ec2f199c1844981a74ee202ad6d2195, 5dbd9f0248d848c5933483a5516ed43c
I20260811 05:17:15.546048 2297 raft_consensus.cc:2726] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:17:15.980383 2298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:15.979769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.129:50493)
0811 05:17:15.980189 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:15.980263 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":130,"threads_started":1}
I20260811 05:17:16.125130 2299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:16.124729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.130:33907)
0811 05:17:16.124983 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:16.125036 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":77,"threads_started":1}
I20260811 05:17:16.326195 2300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:16.325786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.131:60099)
0811 05:17:16.326044 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:16.326098 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":76,"threads_started":1}
I20260811 05:17:16.981663 2301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:16.981234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.129:38355)
0811 05:17:16.981496 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:16.981560 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":77,"threads_started":1}
I20260811 05:17:17.126616 2302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:17.126040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.130:36965)
0811 05:17:17.126437 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:17.126508 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":92,"threads_started":1}
I20260811 05:17:17.327438 2303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:17.326969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.131:42389)
0811 05:17:17.327281 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:17.327347 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":92,"threads_started":1}
I20260811 05:17:17.983037 2305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:17.982582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.129:37851)
0811 05:17:17.982879 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:17.982938 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":91,"threads_started":1}
I20260811 05:17:18.127816 2306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:18.127403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.130:58253)
0811 05:17:18.127667 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:18.127724 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":78,"threads_started":1}
I20260811 05:17:18.328709 2307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:18.328168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.131:57515)
0811 05:17:18.328502 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:18.328616 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":122,"threads_started":1}
I20260811 05:17:18.398727 2308 raft_consensus.cc:482] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:17:18.398908 2308 raft_consensus.cc:504] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } }
I20260811 05:17:18.399379 2308 leader_election.cc:288] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5dbd9f0248d848c5933483a5516ed43c (127.7.67.131:39339), 3ec2f199c1844981a74ee202ad6d2195 (127.7.67.130:38701)
W20260811 05:17:18.400312 1672 leader_election.cc:341] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5dbd9f0248d848c5933483a5516ed43c (127.7.67.131:39339): Not found: Tablet not found: df1d52bc5c76471fadb1dd440d8776d7
W20260811 05:17:18.400434 1676 leader_election.cc:341] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ec2f199c1844981a74ee202ad6d2195 (127.7.67.130:38701): Not found: Tablet not found: df1d52bc5c76471fadb1dd440d8776d7
I20260811 05:17:18.400482 1672 leader_election.cc:302] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16a40bffec56478e9afdaf0b92114754; no voters: 3ec2f199c1844981a74ee202ad6d2195, 5dbd9f0248d848c5933483a5516ed43c
I20260811 05:17:18.400668 2308 raft_consensus.cc:2726] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:17:18.984426 2309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:18.983912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.129:46189)
0811 05:17:18.984235 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:18.984309 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":87,"threads_started":1}
I20260811 05:17:19.129189 2312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:19.128694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.130:38809)
0811 05:17:19.128996 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:19.129068 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":82,"threads_started":1}
I20260811 05:17:19.329870 2314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:19.329466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.131:35597)
0811 05:17:19.329727 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:19.329779 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":83,"threads_started":1}
I20260811 05:17:19.985889 2316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:19.985441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.129:56417)
0811 05:17:19.985723 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:19.985786 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":82,"threads_started":1}
I20260811 05:17:20.130501 2317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:20.130068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.130:58973)
0811 05:17:20.130342 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:20.130405 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":104,"threads_started":1}
I20260811 05:17:20.331143 2318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:20.330698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.131:57591)
0811 05:17:20.330985 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:20.331037 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":121,"threads_started":1}
I20260811 05:17:20.987319 2319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:20.986797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.129:59921)
0811 05:17:20.987093 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:20.987208 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":85,"threads_started":1}
I20260811 05:17:21.065639 2320 raft_consensus.cc:482] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:17:21.065809 2320 raft_consensus.cc:504] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } }
I20260811 05:17:21.066457 2320 leader_election.cc:288] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5dbd9f0248d848c5933483a5516ed43c (127.7.67.131:39339), 3ec2f199c1844981a74ee202ad6d2195 (127.7.67.130:38701)
W20260811 05:17:21.067399 1672 leader_election.cc:341] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5dbd9f0248d848c5933483a5516ed43c (127.7.67.131:39339): Not found: Tablet not found: df1d52bc5c76471fadb1dd440d8776d7
W20260811 05:17:21.067518 1676 leader_election.cc:341] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ec2f199c1844981a74ee202ad6d2195 (127.7.67.130:38701): Not found: Tablet not found: df1d52bc5c76471fadb1dd440d8776d7
I20260811 05:17:21.067598 1676 leader_election.cc:302] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16a40bffec56478e9afdaf0b92114754; no voters: 3ec2f199c1844981a74ee202ad6d2195, 5dbd9f0248d848c5933483a5516ed43c
I20260811 05:17:21.067772 2320 raft_consensus.cc:2726] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:17:21.131753 2321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:21.131345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.130:54839)
0811 05:17:21.131607 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:21.131660 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":89,"threads_started":1}
I20260811 05:17:21.332397 2322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:21.331910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.131:54907)
0811 05:17:21.332223 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:21.332290 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":81,"threads_started":1}
I20260811 05:17:21.988794 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:21.988237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.129:52067)
0811 05:17:21.988601 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:21.988669 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":94,"threads_started":1}
I20260811 05:17:22.133193 2326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:22.132749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.130:60235)
0811 05:17:22.133044 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:22.133099 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":124,"threads_started":1}
I20260811 05:17:22.333681 2328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:22.333197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.131:60133)
0811 05:17:22.333508 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:22.333565 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":100,"threads_started":1}
I20260811 05:17:22.990283 2329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:22.989771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.129:36789)
0811 05:17:22.990105 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:22.990174 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":126,"threads_started":1}
I20260811 05:17:23.134403 2330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:23.133941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.130:37327)
0811 05:17:23.134247 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:23.134303 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":80,"threads_started":1}
I20260811 05:17:23.334982 2332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:23.334511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.131:56557)
0811 05:17:23.334822 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:23.334883 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":101,"threads_started":1}
I20260811 05:17:23.634311 2333 raft_consensus.cc:482] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:17:23.634518 2333 raft_consensus.cc:504] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dbd9f0248d848c5933483a5516ed43c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39339 } } peers { permanent_uuid: "3ec2f199c1844981a74ee202ad6d2195" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 38701 } } peers { permanent_uuid: "16a40bffec56478e9afdaf0b92114754" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 34045 } }
I20260811 05:17:23.635020 2333 leader_election.cc:288] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 5dbd9f0248d848c5933483a5516ed43c (127.7.67.131:39339), 3ec2f199c1844981a74ee202ad6d2195 (127.7.67.130:38701)
W20260811 05:17:23.635847 1676 leader_election.cc:341] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3ec2f199c1844981a74ee202ad6d2195 (127.7.67.130:38701): Not found: Tablet not found: df1d52bc5c76471fadb1dd440d8776d7
W20260811 05:17:23.635942 1672 leader_election.cc:341] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5dbd9f0248d848c5933483a5516ed43c (127.7.67.131:39339): Not found: Tablet not found: df1d52bc5c76471fadb1dd440d8776d7
I20260811 05:17:23.636068 1672 leader_election.cc:302] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 16a40bffec56478e9afdaf0b92114754; no voters: 3ec2f199c1844981a74ee202ad6d2195, 5dbd9f0248d848c5933483a5516ed43c
I20260811 05:17:23.636273 2333 raft_consensus.cc:2726] T df1d52bc5c76471fadb1dd440d8776d7 P 16a40bffec56478e9afdaf0b92114754 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 05:17:23.991662 2334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:23.991198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.129:47379)
0811 05:17:23.991491 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:23.991552 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":83,"threads_started":1}
I20260811 05:17:24.135820 2335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:24.135388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.130:49797)
0811 05:17:24.135658 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:24.135718 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":81,"threads_started":1}
I20260811 05:17:24.336215 2336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:24.335808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.131:50879)
0811 05:17:24.336067 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:24.336122 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":83,"threads_started":1}
I20260811 05:17:24.993276 2337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:24.992805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.129:51861)
0811 05:17:24.993092 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:24.993145 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":95,"threads_started":1}
I20260811 05:17:25.137169 2340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:25.136714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.130:39335)
0811 05:17:25.137001 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:25.137062 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":75,"threads_started":1}
I20260811 05:17:25.329022 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 1588
I20260811 05:17:25.337741 2341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:25.337280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:42737 (local address 127.7.67.131:59961)
0811 05:17:25.337570 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:25.337637 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.190:42737: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":102,"threads_started":1}
I20260811 05:17:25.343245 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 1845
I20260811 05:17:25.353832 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 2055
2026-08-11T05:17:25Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59675 ms)
[----------] 30 tests from SecurityITest (254688 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:25 dist-test-slave-04bb krb5kdc[2344](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:25 dist-test-slave-04bb krb5kdc[2344](info): set up 2 sockets
Aug 11 05:17:25 dist-test-slave-04bb krb5kdc[2344](info): commencing operation
krb5kdc: starting...
W20260811 05:17:26.606309 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.224s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:17:26 dist-test-slave-04bb krb5kdc[2344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:17:26.733683 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:33581
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:45411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:33581
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:26.847703 2370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:26.848014 2370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:26.848093 2370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:26.852051 2370 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:26.852146 2370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:26.852182 2370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:26.852221 2370 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:26.852257 2370 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:26.857524 2370 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:45411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:33581
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:33581
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:26.858665 2370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:26.859745 2370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:26.868686 2378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:26.868649 2381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:26.868641 2377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:26.869029 2370 server_base.cc:1034] running on GCE node
I20260811 05:17:26.869576 2370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:26.870702 2370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:26.871928 2370 hybrid_clock.cc:648] HybridClock initialized: now 1786425446871911 us; error 27 us; skew 500 ppm
Aug 11 05:17:26 dist-test-slave-04bb krb5kdc[2344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425446, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:26.878410 2370 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:17:26.879441 2370 webserver.cc:460] Webserver started at http://127.7.67.190:46143/ using document root <none> and password file <none>
I20260811 05:17:26.880059 2370 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:26.880154 2370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:26.880406 2370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:26.882577 2370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "9c358ad7e13543c6b6b08f8d4bf6e8ad"
format_stamp: "Formatted at 2026-08-11 05:17:26 on dist-test-slave-04bb"
server_key: "b562d412f655d9f0a9272cc4922540f9"
server_key_iv: "ab80093784777e9e3e0957e7e473f310"
server_key_version: "encryptionkey@0"
I20260811 05:17:26.883150 2370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "9c358ad7e13543c6b6b08f8d4bf6e8ad"
format_stamp: "Formatted at 2026-08-11 05:17:26 on dist-test-slave-04bb"
server_key: "b562d412f655d9f0a9272cc4922540f9"
server_key_iv: "ab80093784777e9e3e0957e7e473f310"
server_key_version: "encryptionkey@0"
I20260811 05:17:26.887089 2370 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 05:17:26.889647 2388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:26.890980 2370 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:17:26.891187 2370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "9c358ad7e13543c6b6b08f8d4bf6e8ad"
format_stamp: "Formatted at 2026-08-11 05:17:26 on dist-test-slave-04bb"
server_key: "b562d412f655d9f0a9272cc4922540f9"
server_key_iv: "ab80093784777e9e3e0957e7e473f310"
server_key_version: "encryptionkey@0"
I20260811 05:17:26.891314 2370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:26.908164 2370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:26.911482 2370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:26.911700 2370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:26.919252 2370 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:33581
I20260811 05:17:26.919277 2463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:33581 every 8 connection(s)
I20260811 05:17:26.920262 2370 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:26.923090 2464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:26.928802 2464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c358ad7e13543c6b6b08f8d4bf6e8ad: Bootstrap starting.
I20260811 05:17:26.930338 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 2370
I20260811 05:17:26.930471 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:17:26.930783 7438 external_mini_cluster.cc:1428] Setting key 9f48fe38dc7ff3da830d06eeb80f6ad3
I20260811 05:17:26.931734 2464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c358ad7e13543c6b6b08f8d4bf6e8ad: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:26.932772 2464 log.cc:824] T 00000000000000000000000000000000 P 9c358ad7e13543c6b6b08f8d4bf6e8ad: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:26.934800 2464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c358ad7e13543c6b6b08f8d4bf6e8ad: No bootstrap required, opened a new log
I20260811 05:17:26.938644 2464 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c358ad7e13543c6b6b08f8d4bf6e8ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c358ad7e13543c6b6b08f8d4bf6e8ad" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 33581 } }
I20260811 05:17:26.938898 2464 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c358ad7e13543c6b6b08f8d4bf6e8ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:26.938995 2464 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c358ad7e13543c6b6b08f8d4bf6e8ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c358ad7e13543c6b6b08f8d4bf6e8ad, State: Initialized, Role: FOLLOWER
I20260811 05:17:26.939498 2464 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c358ad7e13543c6b6b08f8d4bf6e8ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c358ad7e13543c6b6b08f8d4bf6e8ad" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 33581 } }
I20260811 05:17:26.939668 2464 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c358ad7e13543c6b6b08f8d4bf6e8ad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:26.939744 2464 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c358ad7e13543c6b6b08f8d4bf6e8ad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:26.939867 2464 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c358ad7e13543c6b6b08f8d4bf6e8ad [term 0 FOLLOWER]: Advancing to term 1
Aug 11 05:17:26 dist-test-slave-04bb krb5kdc[2344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:17:26.940848 2464 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c358ad7e13543c6b6b08f8d4bf6e8ad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c358ad7e13543c6b6b08f8d4bf6e8ad" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 33581 } }
I20260811 05:17:26.941256 2464 leader_election.cc:302] T 00000000000000000000000000000000 P 9c358ad7e13543c6b6b08f8d4bf6e8ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c358ad7e13543c6b6b08f8d4bf6e8ad; no voters:
I20260811 05:17:26.941632 2464 leader_election.cc:288] T 00000000000000000000000000000000 P 9c358ad7e13543c6b6b08f8d4bf6e8ad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:26.941751 2471 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c358ad7e13543c6b6b08f8d4bf6e8ad [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:26.942113 2471 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c358ad7e13543c6b6b08f8d4bf6e8ad [term 1 LEADER]: Becoming Leader. State: Replica: 9c358ad7e13543c6b6b08f8d4bf6e8ad, State: Running, Role: LEADER
I20260811 05:17:26.942492 2471 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c358ad7e13543c6b6b08f8d4bf6e8ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c358ad7e13543c6b6b08f8d4bf6e8ad" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 33581 } }
I20260811 05:17:26.943042 2464 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c358ad7e13543c6b6b08f8d4bf6e8ad [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:26.944244 2472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c358ad7e13543c6b6b08f8d4bf6e8ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c358ad7e13543c6b6b08f8d4bf6e8ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c358ad7e13543c6b6b08f8d4bf6e8ad" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 33581 } } }
I20260811 05:17:26.944368 2472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c358ad7e13543c6b6b08f8d4bf6e8ad [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:26.944612 2472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c358ad7e13543c6b6b08f8d4bf6e8ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c358ad7e13543c6b6b08f8d4bf6e8ad. Latest consensus state: current_term: 1 leader_uuid: "9c358ad7e13543c6b6b08f8d4bf6e8ad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c358ad7e13543c6b6b08f8d4bf6e8ad" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 33581 } } }
I20260811 05:17:26.944716 2472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c358ad7e13543c6b6b08f8d4bf6e8ad [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:26.945204 2481 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:26.946965 2469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:26.932062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46604 (local address 127.7.67.190:33581)
0811 05:17:26.932621 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:26.932637 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:26.932675 (+ 38us) server_negotiation.cc:407] Connection header received
0811 05:17:26.933503 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:26.933530 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:26.933899 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:26.934236 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:26.935079 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:26.936091 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:26.937010 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:26.937484 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:26.937660 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:26.940630 (+ 2970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:26.940636 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:26.940650 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:26.940686 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:26.942889 (+ 2203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:26.943417 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:26.943423 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:26.943429 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:26.943587 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:26.943976 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:26.943982 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:26.943985 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:26.944625 (+ 640us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:26.945526 (+ 901us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:26.945697 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 05:17:26.946001 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":140,"threads_started":1}
I20260811 05:17:26.949208 2481 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:26.955040 2481 catalog_manager.cc:1347] Generated new cluster ID: f44b070ea1144f9a8c4b6b805fdb59b3
I20260811 05:17:26.955104 2481 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:26.987388 2481 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:26.988812 2481 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:26.997143 2481 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c358ad7e13543c6b6b08f8d4bf6e8ad: Generated new TSK 0
I20260811 05:17:26.997972 2481 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
I20260811 05:17:27.039331 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:33581
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:45411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:27.154143 2499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:27.154397 2499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:27.154441 2499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:27.158480 2499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:27.158531 2499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:27.158588 2499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:17:27.163759 2499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:45411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:33581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:27.164741 2499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:27.165782 2499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:27.175501 2506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:27.175647 2508 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:27.175704 2499 server_base.cc:1034] running on GCE node
W20260811 05:17:27.175547 2505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:27.176133 2499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:27.176805 2499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:27.177965 2499 hybrid_clock.cc:648] HybridClock initialized: now 1786425447177950 us; error 27 us; skew 500 ppm
Aug 11 05:17:27 dist-test-slave-04bb krb5kdc[2344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425447, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:27.180799 2499 init.cc:377] Logged in from keytab as kudu/127.7.67.129@KRBTEST.COM (short username kudu)
I20260811 05:17:27.181713 2499 webserver.cc:460] Webserver started at http://127.7.67.129:34529/ using document root <none> and password file <none>
I20260811 05:17:27.182291 2499 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:27.182384 2499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:27.182618 2499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:27.184708 2499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "e507948f786346c1b434523d0eb94b99"
format_stamp: "Formatted at 2026-08-11 05:17:27 on dist-test-slave-04bb"
server_key: "192a20798311903ee358a60ff9c56797"
server_key_iv: "acaf035df3c29c9154351a3e0870a13c"
server_key_version: "encryptionkey@0"
I20260811 05:17:27.185293 2499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "e507948f786346c1b434523d0eb94b99"
format_stamp: "Formatted at 2026-08-11 05:17:27 on dist-test-slave-04bb"
server_key: "192a20798311903ee358a60ff9c56797"
server_key_iv: "acaf035df3c29c9154351a3e0870a13c"
server_key_version: "encryptionkey@0"
I20260811 05:17:27.188927 2499 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:17:27.191264 2517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:27.192478 2499 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:17:27.192642 2499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "e507948f786346c1b434523d0eb94b99"
format_stamp: "Formatted at 2026-08-11 05:17:27 on dist-test-slave-04bb"
server_key: "192a20798311903ee358a60ff9c56797"
server_key_iv: "acaf035df3c29c9154351a3e0870a13c"
server_key_version: "encryptionkey@0"
I20260811 05:17:27.192752 2499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:27.207101 2499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:27.210260 2499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:27.210472 2499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:27.211057 2499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:27.211882 2499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:27.211954 2499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:27.212026 2499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:27.212078 2499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:27.221249 2499 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:46053
I20260811 05:17:27.221282 2698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:46053 every 8 connection(s)
I20260811 05:17:27.222107 2499 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:17:27.226478 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 2499
I20260811 05:17:27.226629 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:17:27.226922 7438 external_mini_cluster.cc:1428] Setting key 33000a53a93bba14c9728c25d3ef4dbd
Aug 11 05:17:27 dist-test-slave-04bb krb5kdc[2344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425447, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:17:27.236410 2469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:27.223938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:42325 (local address 127.7.67.190:33581)
0811 05:17:27.224078 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:27.224082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:27.224910 (+ 828us) server_negotiation.cc:407] Connection header received
0811 05:17:27.226305 (+ 1395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:27.226308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:27.226369 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:27.226474 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:27.227858 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.228513 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.229703 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.229951 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.230061 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:27.232578 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:27.232586 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:27.232589 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:27.232612 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:27.234154 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:27.234814 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:27.234817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:27.234818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:27.234890 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:27.235301 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:27.235303 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:27.235304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:27.235463 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:27.235537 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:27.236167 (+ 630us) server_negotiation.cc:299] Negotiation successful
0811 05:17:27.236290 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 05:17:27.237603 2703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:27.224246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:33581 (local address 127.7.67.129:42325)
0811 05:17:27.224759 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:27.224801 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:27.226049 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:27.226665 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:27.226672 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:27.227113 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:27.227694 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:27.227710 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.228705 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.228707 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:27.229575 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:27.229584 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.230112 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.230116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:27.230289 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:27.230903 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:27.230920 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:27.232390 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:27.234282 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:27.234290 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:27.234309 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:27.234709 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:27.234990 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:27.234995 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:27.235000 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:27.235198 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:27.235570 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:27.235577 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:27.235898 (+ 321us) client_negotiation.cc:769] Sending connection context
0811 05:17:27.236131 (+ 233us) client_negotiation.cc:240] Negotiation successful
0811 05:17:27.236485 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":123,"threads_started":1}
WARNING: no policy specified for kudu/127.7.67.130@KRBTEST.COM; defaulting to no policy
I20260811 05:17:27.239399 2699 heartbeater.cc:347] Connected to a master server at 127.7.67.190:33581
I20260811 05:17:27.239656 2699 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:27.240197 2699 heartbeater.cc:511] Master 127.7.67.190:33581 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.130@KRBTEST.COM" created.
I20260811 05:17:27.241863 2416 ts_manager.cc:194] Registered new tserver with Master: e507948f786346c1b434523d0eb94b99 (127.7.67.129:46053)
I20260811 05:17:27.243532 2416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.129@KRBTEST.COM'} at 127.7.67.129:42325
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
I20260811 05:17:27.257800 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:33581
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:45411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:27.372771 2711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:27.373021 2711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:27.373063 2711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:27.377044 2711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:27.377099 2711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:27.377154 2711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:17:27.382513 2711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:45411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:33581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:27.383617 2711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:27.384747 2711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:27.394443 2720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:27.394613 2711 server_base.cc:1034] running on GCE node
W20260811 05:17:27.394544 2717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:27.394510 2718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:27.395159 2711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:27.395910 2711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:27.397102 2711 hybrid_clock.cc:648] HybridClock initialized: now 1786425447397065 us; error 48 us; skew 500 ppm
Aug 11 05:17:27 dist-test-slave-04bb krb5kdc[2344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425447, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:27.399852 2711 init.cc:377] Logged in from keytab as kudu/127.7.67.130@KRBTEST.COM (short username kudu)
I20260811 05:17:27.400815 2711 webserver.cc:460] Webserver started at http://127.7.67.130:34979/ using document root <none> and password file <none>
I20260811 05:17:27.401392 2711 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:27.401484 2711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:27.401722 2711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:27.403708 2711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "a94be11c46444055b3afff63d2043a02"
format_stamp: "Formatted at 2026-08-11 05:17:27 on dist-test-slave-04bb"
server_key: "44719dcb08e9a2104ae6f9cbf0a11df5"
server_key_iv: "33db54771dbd733eac8446645d4543c0"
server_key_version: "encryptionkey@0"
I20260811 05:17:27.404285 2711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "a94be11c46444055b3afff63d2043a02"
format_stamp: "Formatted at 2026-08-11 05:17:27 on dist-test-slave-04bb"
server_key: "44719dcb08e9a2104ae6f9cbf0a11df5"
server_key_iv: "33db54771dbd733eac8446645d4543c0"
server_key_version: "encryptionkey@0"
I20260811 05:17:27.407932 2711 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 05:17:27.410401 2730 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:27.411700 2711 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 05:17:27.411859 2711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "a94be11c46444055b3afff63d2043a02"
format_stamp: "Formatted at 2026-08-11 05:17:27 on dist-test-slave-04bb"
server_key: "44719dcb08e9a2104ae6f9cbf0a11df5"
server_key_iv: "33db54771dbd733eac8446645d4543c0"
server_key_version: "encryptionkey@0"
I20260811 05:17:27.411976 2711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:27.427511 2711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:27.430513 2711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:27.430732 2711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:27.431339 2711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:27.432184 2711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:27.432262 2711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:27.432354 2711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:27.432404 2711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:27.441507 2711 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:44817
I20260811 05:17:27.441538 2922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:44817 every 8 connection(s)
I20260811 05:17:27.442406 2711 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:17:27.444748 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 2711
I20260811 05:17:27.444917 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:17:27.445286 7438 external_mini_cluster.cc:1428] Setting key 6e5bb7e122c3883a60ccd3e1da8b37df
Aug 11 05:17:27 dist-test-slave-04bb krb5kdc[2344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425447, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.131@KRBTEST.COM; defaulting to no policy
I20260811 05:17:27.458432 2469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:27.444226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:42361 (local address 127.7.67.190:33581)
0811 05:17:27.444392 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:27.444401 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:27.445715 (+ 1314us) server_negotiation.cc:407] Connection header received
0811 05:17:27.447403 (+ 1688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:27.447406 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:27.447450 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:27.447519 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:27.449223 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.450087 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.451253 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.451673 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.451799 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:27.454392 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:27.454399 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:27.454405 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:27.454436 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:27.456150 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:27.456745 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:27.456750 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:27.456754 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:27.456856 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:27.457247 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:27.457251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:27.457252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:27.457469 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:27.457554 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:27.458184 (+ 630us) server_negotiation.cc:299] Negotiation successful
0811 05:17:27.458301 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.131@KRBTEST.COM" created.
I20260811 05:17:27.459318 2926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:27.444775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:33581 (local address 127.7.67.130:42361)
0811 05:17:27.445303 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:27.445376 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:27.447151 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:27.447697 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:27.447709 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:27.448281 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:27.449029 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:27.449054 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.450227 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.450248 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:27.451119 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:27.451129 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.451828 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.451831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:27.451977 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:27.452705 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:27.452722 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:27.454201 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:27.456279 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:27.456284 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:27.456297 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:27.456622 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:27.456957 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:27.456959 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:27.456960 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:27.457097 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:27.457577 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:27.457582 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:27.457893 (+ 311us) client_negotiation.cc:769] Sending connection context
0811 05:17:27.458060 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 05:17:27.458346 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":145,"threads_started":1}
I20260811 05:17:27.460598 2923 heartbeater.cc:347] Connected to a master server at 127.7.67.190:33581
I20260811 05:17:27.460882 2923 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:27.461509 2923 heartbeater.cc:511] Master 127.7.67.190:33581 requested a full tablet report, sending...
I20260811 05:17:27.462723 2416 ts_manager.cc:194] Registered new tserver with Master: a94be11c46444055b3afff63d2043a02 (127.7.67.130:44817)
I20260811 05:17:27.463404 2416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.130@KRBTEST.COM'} at 127.7.67.130:42361
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
I20260811 05:17:27.477600 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:33581
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:45411
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:27.593174 2931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:27.593438 2931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:27.593505 2931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:27.597746 2931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:27.597824 2931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:27.597903 2931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:17:27.603051 2931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:45411
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:33581
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:27.604065 2931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:27.605144 2931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:27.614856 2939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:27.614866 2944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:27.615065 2931 server_base.cc:1034] running on GCE node
W20260811 05:17:27.615249 2942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:27.615658 2931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:27.616372 2931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:27.617559 2931 hybrid_clock.cc:648] HybridClock initialized: now 1786425447617538 us; error 39 us; skew 500 ppm
Aug 11 05:17:27 dist-test-slave-04bb krb5kdc[2344](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425447, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:27.620253 2931 init.cc:377] Logged in from keytab as kudu/127.7.67.131@KRBTEST.COM (short username kudu)
I20260811 05:17:27.621165 2931 webserver.cc:460] Webserver started at http://127.7.67.131:43165/ using document root <none> and password file <none>
I20260811 05:17:27.621730 2931 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:27.621824 2931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:27.622053 2931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:27.624059 2931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "f38f4b41425747a59ed2a1807ccb79cb"
format_stamp: "Formatted at 2026-08-11 05:17:27 on dist-test-slave-04bb"
server_key: "3a0d6c3125f64d63b8fff6d6dbc0bcde"
server_key_iv: "6069a6b1bd62adaf4eefde62eaca05bc"
server_key_version: "encryptionkey@0"
I20260811 05:17:27.624675 2931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "f38f4b41425747a59ed2a1807ccb79cb"
format_stamp: "Formatted at 2026-08-11 05:17:27 on dist-test-slave-04bb"
server_key: "3a0d6c3125f64d63b8fff6d6dbc0bcde"
server_key_iv: "6069a6b1bd62adaf4eefde62eaca05bc"
server_key_version: "encryptionkey@0"
I20260811 05:17:27.628278 2931 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:17:27.630689 2955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:27.631861 2931 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 05:17:27.631992 2931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "f38f4b41425747a59ed2a1807ccb79cb"
format_stamp: "Formatted at 2026-08-11 05:17:27 on dist-test-slave-04bb"
server_key: "3a0d6c3125f64d63b8fff6d6dbc0bcde"
server_key_iv: "6069a6b1bd62adaf4eefde62eaca05bc"
server_key_version: "encryptionkey@0"
I20260811 05:17:27.632097 2931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:27.659616 2931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:27.663069 2931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:27.663332 2931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:27.664112 2931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:27.665088 2931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:27.665166 2931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:27.665233 2931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:27.665288 2931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:27.674993 2931 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:42267
I20260811 05:17:27.675047 3123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:42267 every 8 connection(s)
I20260811 05:17:27.675989 2931 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:17:27.685323 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 2931
I20260811 05:17:27.685501 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:17:27.685916 7438 external_mini_cluster.cc:1428] Setting key 1027461b0fdc674992d5dcfcf1ea96f4
Aug 11 05:17:27 dist-test-slave-04bb krb5kdc[2344](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425447, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:17:27.690984 2469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:27.677939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:42549 (local address 127.7.67.190:33581)
0811 05:17:27.678098 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:27.678103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:27.679039 (+ 936us) server_negotiation.cc:407] Connection header received
0811 05:17:27.680475 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:27.680478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:27.680573 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:27.680653 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:27.681848 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.682395 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.683603 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.683829 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.683910 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:27.687084 (+ 3174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:27.687088 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:27.687091 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:27.687123 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:27.689027 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:27.689554 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:27.689556 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:27.689558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:27.689645 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:27.690011 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:27.690014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:27.690015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:27.690206 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:27.690295 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:27.690747 (+ 452us) server_negotiation.cc:299] Negotiation successful
0811 05:17:27.690863 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 05:17:27.691900 3128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:27.678388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:33581 (local address 127.7.67.131:42549)
0811 05:17:27.678873 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:27.678919 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:27.680242 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:27.680806 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:27.680813 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:27.681194 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:27.681714 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:27.681729 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.682533 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.682535 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:27.683491 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:27.683501 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.683970 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.683973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:27.684124 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:27.684776 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:27.684794 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:27.686863 (+ 2069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:27.689162 (+ 2299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:27.689167 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:27.689180 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:27.689455 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:27.689758 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:27.689760 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:27.689761 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:27.689893 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:27.690305 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:27.690311 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:27.690539 (+ 228us) client_negotiation.cc:769] Sending connection context
0811 05:17:27.690744 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 05:17:27.691113 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":121,"threads_started":1}
I20260811 05:17:27.693073 3124 heartbeater.cc:347] Connected to a master server at 127.7.67.190:33581
I20260811 05:17:27.693324 3124 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:27.693835 3124 heartbeater.cc:511] Master 127.7.67.190:33581 requested a full tablet report, sending...
I20260811 05:17:27.694967 2416 ts_manager.cc:194] Registered new tserver with Master: f38f4b41425747a59ed2a1807ccb79cb (127.7.67.131:42267)
I20260811 05:17:27.695634 2416 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.131@KRBTEST.COM'} at 127.7.67.131:42549
I20260811 05:17:27.700395 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:17:27.710072 2469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:27.702537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46616 (local address 127.7.67.190:33581)
0811 05:17:27.702654 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:27.702657 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:27.702839 (+ 182us) server_negotiation.cc:407] Connection header received
0811 05:17:27.702933 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:27.702935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:27.702972 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:27.703041 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:27.703985 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.704485 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.705375 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.705579 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.705661 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:27.706972 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:27.706975 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:27.706977 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:27.706999 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:27.708460 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:27.708893 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:27.708896 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:27.708897 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:27.708972 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:27.709291 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:27.709297 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:27.709300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:27.709530 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:27.709640 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:27.709859 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 05:17:27.709954 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 05:17:27.712992 2416 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46616:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:17:27.716208 2416 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:17:27.731073 3144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:27.725328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43252 (local address 127.7.67.129:46053)
0811 05:17:27.726306 (+ 978us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:27.726319 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:27.726338 (+ 19us) server_negotiation.cc:407] Connection header received
0811 05:17:27.726437 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:27.726443 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:27.726664 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:27.726848 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:27.727265 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.728194 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.729741 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.730323 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.730454 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:27.730617 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:27.730808 (+ 191us) server_negotiation.cc:299] Negotiation successful
0811 05:17:27.730907 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":881,"thread_start_us":126,"threads_started":1}
I20260811 05:17:27.731058 3142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:27.725037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:46053 (local address 127.0.0.1:43252)
0811 05:17:27.725908 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:27.725962 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:27.726101 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:27.726897 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:27.726904 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:27.726924 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:27.727073 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:27.727085 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.728341 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.728344 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:27.729607 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:27.729617 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.730459 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.730462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:27.730579 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:27.730619 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 05:17:27.730762 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 05:17:27.730891 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"mutex_wait_us":4,"thread_start_us":99,"threads_started":1}
I20260811 05:17:27.732154 3145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:27.725458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:42267 (local address 127.0.0.1:35732)
0811 05:17:27.726659 (+ 1201us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:27.726674 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:27.726774 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:27.727358 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:27.727361 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:27.727381 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:27.727459 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:27.727465 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.728876 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.728879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:27.730416 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:27.730428 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.731732 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.731735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:27.731829 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:27.731854 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 05:17:27.731929 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 05:17:27.732009 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1133,"mutex_wait_us":152,"thread_start_us":88,"threads_started":1}
I20260811 05:17:27.732481 3146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:27.725712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40786 (local address 127.7.67.130:44817)
0811 05:17:27.726126 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:27.726139 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:27.726726 (+ 587us) server_negotiation.cc:407] Connection header received
0811 05:17:27.726814 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:27.726822 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:27.727006 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:27.727168 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:27.727446 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.728702 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.730562 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.731489 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.731602 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:27.731736 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:27.732227 (+ 491us) server_negotiation.cc:299] Negotiation successful
0811 05:17:27.732327 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":113,"threads_started":1}
I20260811 05:17:27.732676 3147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:27.726014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35732 (local address 127.7.67.131:42267)
0811 05:17:27.726513 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:27.726530 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:27.726779 (+ 249us) server_negotiation.cc:407] Connection header received
0811 05:17:27.726869 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:27.726874 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:27.727062 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:27.727227 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:27.727582 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.728745 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.730562 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.731528 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.731617 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:27.731736 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:27.732393 (+ 657us) server_negotiation.cc:299] Negotiation successful
0811 05:17:27.732489 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":111,"threads_started":1}
I20260811 05:17:27.732791 3141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:27.725118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:44817 (local address 127.0.0.1:40786)
0811 05:17:27.726480 (+ 1362us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:27.726495 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:27.726598 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:27.727163 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:27.727168 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:27.727190 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:27.727299 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:27.727306 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.728876 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.728879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:27.730416 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:27.730428 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.731812 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.731815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:27.731932 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:27.731955 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 05:17:27.732032 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 05:17:27.732114 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1297,"mutex_wait_us":199,"thread_start_us":116,"threads_started":1}
I20260811 05:17:27.734530 2597 tablet_service.cc:1467] Processing CreateTablet for tablet e2ee9276a8724deeb0e85b9fa76350a5 (DEFAULT_TABLE table=test-table [id=e8dd7c22a67f4c7886e74b8af408e6b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:27.735008 3022 tablet_service.cc:1467] Processing CreateTablet for tablet e2ee9276a8724deeb0e85b9fa76350a5 (DEFAULT_TABLE table=test-table [id=e8dd7c22a67f4c7886e74b8af408e6b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:27.735783 2597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2ee9276a8724deeb0e85b9fa76350a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:27.736109 3022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2ee9276a8724deeb0e85b9fa76350a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:27.737136 2823 tablet_service.cc:1467] Processing CreateTablet for tablet e2ee9276a8724deeb0e85b9fa76350a5 (DEFAULT_TABLE table=test-table [id=e8dd7c22a67f4c7886e74b8af408e6b5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:27.738199 2823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2ee9276a8724deeb0e85b9fa76350a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:27.742681 3148 tablet_bootstrap.cc:492] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb: Bootstrap starting.
I20260811 05:17:27.744078 3152 tablet_bootstrap.cc:492] T e2ee9276a8724deeb0e85b9fa76350a5 P e507948f786346c1b434523d0eb94b99: Bootstrap starting.
I20260811 05:17:27.745121 3148 tablet_bootstrap.cc:654] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:27.745519 3153 tablet_bootstrap.cc:492] T e2ee9276a8724deeb0e85b9fa76350a5 P a94be11c46444055b3afff63d2043a02: Bootstrap starting.
I20260811 05:17:27.746285 3152 tablet_bootstrap.cc:654] T e2ee9276a8724deeb0e85b9fa76350a5 P e507948f786346c1b434523d0eb94b99: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:27.746949 3152 log.cc:824] T e2ee9276a8724deeb0e85b9fa76350a5 P e507948f786346c1b434523d0eb94b99: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:27.747229 3148 log.cc:824] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:27.747973 3153 tablet_bootstrap.cc:654] T e2ee9276a8724deeb0e85b9fa76350a5 P a94be11c46444055b3afff63d2043a02: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:27.748900 3152 tablet_bootstrap.cc:492] T e2ee9276a8724deeb0e85b9fa76350a5 P e507948f786346c1b434523d0eb94b99: No bootstrap required, opened a new log
I20260811 05:17:27.749104 3152 ts_tablet_manager.cc:1397] T e2ee9276a8724deeb0e85b9fa76350a5 P e507948f786346c1b434523d0eb94b99: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:17:27.749325 3153 log.cc:824] T e2ee9276a8724deeb0e85b9fa76350a5 P a94be11c46444055b3afff63d2043a02: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:27.749505 3148 tablet_bootstrap.cc:492] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb: No bootstrap required, opened a new log
I20260811 05:17:27.749715 3148 ts_tablet_manager.cc:1397] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 05:17:27.751296 3153 tablet_bootstrap.cc:492] T e2ee9276a8724deeb0e85b9fa76350a5 P a94be11c46444055b3afff63d2043a02: No bootstrap required, opened a new log
I20260811 05:17:27.751539 3153 ts_tablet_manager.cc:1397] T e2ee9276a8724deeb0e85b9fa76350a5 P a94be11c46444055b3afff63d2043a02: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.004s
I20260811 05:17:27.752990 3152 raft_consensus.cc:348] T e2ee9276a8724deeb0e85b9fa76350a5 P e507948f786346c1b434523d0eb94b99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507948f786346c1b434523d0eb94b99" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46053 } } peers { permanent_uuid: "a94be11c46444055b3afff63d2043a02" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44817 } } peers { permanent_uuid: "f38f4b41425747a59ed2a1807ccb79cb" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 42267 } }
I20260811 05:17:27.753203 3152 raft_consensus.cc:374] T e2ee9276a8724deeb0e85b9fa76350a5 P e507948f786346c1b434523d0eb94b99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:27.753250 3152 raft_consensus.cc:729] T e2ee9276a8724deeb0e85b9fa76350a5 P e507948f786346c1b434523d0eb94b99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e507948f786346c1b434523d0eb94b99, State: Initialized, Role: FOLLOWER
I20260811 05:17:27.753659 3152 consensus_queue.cc:260] T e2ee9276a8724deeb0e85b9fa76350a5 P e507948f786346c1b434523d0eb94b99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507948f786346c1b434523d0eb94b99" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46053 } } peers { permanent_uuid: "a94be11c46444055b3afff63d2043a02" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44817 } } peers { permanent_uuid: "f38f4b41425747a59ed2a1807ccb79cb" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 42267 } }
I20260811 05:17:27.753590 3148 raft_consensus.cc:348] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507948f786346c1b434523d0eb94b99" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46053 } } peers { permanent_uuid: "a94be11c46444055b3afff63d2043a02" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44817 } } peers { permanent_uuid: "f38f4b41425747a59ed2a1807ccb79cb" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 42267 } }
I20260811 05:17:27.753880 3148 raft_consensus.cc:374] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:27.753939 3148 raft_consensus.cc:729] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f38f4b41425747a59ed2a1807ccb79cb, State: Initialized, Role: FOLLOWER
I20260811 05:17:27.754351 3148 consensus_queue.cc:260] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507948f786346c1b434523d0eb94b99" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46053 } } peers { permanent_uuid: "a94be11c46444055b3afff63d2043a02" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44817 } } peers { permanent_uuid: "f38f4b41425747a59ed2a1807ccb79cb" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 42267 } }
I20260811 05:17:27.754439 2699 heartbeater.cc:503] Master 127.7.67.190:33581 was elected leader, sending a full tablet report...
I20260811 05:17:27.754681 3152 ts_tablet_manager.cc:1428] T e2ee9276a8724deeb0e85b9fa76350a5 P e507948f786346c1b434523d0eb94b99: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 05:17:27.755236 3124 heartbeater.cc:503] Master 127.7.67.190:33581 was elected leader, sending a full tablet report...
I20260811 05:17:27.755349 3148 ts_tablet_manager.cc:1428] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 05:17:27.755719 3153 raft_consensus.cc:348] T e2ee9276a8724deeb0e85b9fa76350a5 P a94be11c46444055b3afff63d2043a02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507948f786346c1b434523d0eb94b99" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46053 } } peers { permanent_uuid: "a94be11c46444055b3afff63d2043a02" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44817 } } peers { permanent_uuid: "f38f4b41425747a59ed2a1807ccb79cb" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 42267 } }
I20260811 05:17:27.756021 3153 raft_consensus.cc:374] T e2ee9276a8724deeb0e85b9fa76350a5 P a94be11c46444055b3afff63d2043a02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:27.756098 3153 raft_consensus.cc:729] T e2ee9276a8724deeb0e85b9fa76350a5 P a94be11c46444055b3afff63d2043a02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a94be11c46444055b3afff63d2043a02, State: Initialized, Role: FOLLOWER
I20260811 05:17:27.756632 3153 consensus_queue.cc:260] T e2ee9276a8724deeb0e85b9fa76350a5 P a94be11c46444055b3afff63d2043a02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507948f786346c1b434523d0eb94b99" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46053 } } peers { permanent_uuid: "a94be11c46444055b3afff63d2043a02" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44817 } } peers { permanent_uuid: "f38f4b41425747a59ed2a1807ccb79cb" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 42267 } }
I20260811 05:17:27.757296 2923 heartbeater.cc:503] Master 127.7.67.190:33581 was elected leader, sending a full tablet report...
I20260811 05:17:27.757684 3153 ts_tablet_manager.cc:1428] T e2ee9276a8724deeb0e85b9fa76350a5 P a94be11c46444055b3afff63d2043a02: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:17:27.852895 3159 raft_consensus.cc:482] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:17:27.853145 3159 raft_consensus.cc:504] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507948f786346c1b434523d0eb94b99" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46053 } } peers { permanent_uuid: "a94be11c46444055b3afff63d2043a02" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44817 } } peers { permanent_uuid: "f38f4b41425747a59ed2a1807ccb79cb" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 42267 } }
I20260811 05:17:27.854455 3159 leader_election.cc:288] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e507948f786346c1b434523d0eb94b99 (127.7.67.129:46053), a94be11c46444055b3afff63d2043a02 (127.7.67.130:44817)
I20260811 05:17:27.859433 3146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:27.854980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:52329 (local address 127.7.67.130:44817)
0811 05:17:27.855102 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:27.855106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:27.855331 (+ 225us) server_negotiation.cc:407] Connection header received
0811 05:17:27.855401 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:27.855403 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:27.855443 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:27.855504 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:27.855790 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.856546 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.858110 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.858752 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.858889 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:27.858947 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:27.859220 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 05:17:27.859316 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:17:27.859433 3144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:27.854871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:58695 (local address 127.7.67.129:46053)
0811 05:17:27.855005 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:27.855010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:27.855030 (+ 20us) server_negotiation.cc:407] Connection header received
0811 05:17:27.855100 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:27.855103 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:27.855164 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:27.855233 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:27.855599 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.856413 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.858110 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.858752 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.858890 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:27.858947 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:27.859220 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 05:17:27.859296 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:17:27.859444 3128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:27.854711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:46053 (local address 127.7.67.131:58695)
0811 05:17:27.854877 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:27.854897 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:27.854996 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:27.855284 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:27.855286 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:27.855307 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:27.855448 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:27.855453 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.856592 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.856613 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:27.857956 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:27.857969 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.858917 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.858921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:27.859086 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:27.859101 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 05:17:27.859225 (+ 124us) client_negotiation.cc:240] Negotiation successful
0811 05:17:27.859281 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":42}
I20260811 05:17:27.859454 3165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:27.854742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:44817 (local address 127.7.67.131:52329)
0811 05:17:27.855193 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:27.855204 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:27.855322 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:27.855541 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:27.855544 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:27.855565 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:27.855674 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:27.855678 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.856649 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.856652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:27.857958 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:27.857968 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.858917 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.858924 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:27.859069 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:27.859098 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 05:17:27.859199 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 05:17:27.859280 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":70,"spinlock_wait_cycles":3456,"thread_start_us":154,"threads_started":1}
I20260811 05:17:27.860409 2856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2ee9276a8724deeb0e85b9fa76350a5" candidate_uuid: "f38f4b41425747a59ed2a1807ccb79cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a94be11c46444055b3afff63d2043a02" is_pre_election: true
I20260811 05:17:27.860412 2629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2ee9276a8724deeb0e85b9fa76350a5" candidate_uuid: "f38f4b41425747a59ed2a1807ccb79cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e507948f786346c1b434523d0eb94b99" is_pre_election: true
I20260811 05:17:27.860800 2629 raft_consensus.cc:2454] T e2ee9276a8724deeb0e85b9fa76350a5 P e507948f786346c1b434523d0eb94b99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f38f4b41425747a59ed2a1807ccb79cb in term 0.
I20260811 05:17:27.860800 2856 raft_consensus.cc:2454] T e2ee9276a8724deeb0e85b9fa76350a5 P a94be11c46444055b3afff63d2043a02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f38f4b41425747a59ed2a1807ccb79cb in term 0.
I20260811 05:17:27.861387 2960 leader_election.cc:302] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a94be11c46444055b3afff63d2043a02, e507948f786346c1b434523d0eb94b99, f38f4b41425747a59ed2a1807ccb79cb; no voters:
I20260811 05:17:27.861722 3159 raft_consensus.cc:2781] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:17:27.861829 3159 raft_consensus.cc:482] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:17:27.861913 3159 raft_consensus.cc:3037] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:27.863029 3159 raft_consensus.cc:504] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507948f786346c1b434523d0eb94b99" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46053 } } peers { permanent_uuid: "a94be11c46444055b3afff63d2043a02" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44817 } } peers { permanent_uuid: "f38f4b41425747a59ed2a1807ccb79cb" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 42267 } }
I20260811 05:17:27.863421 3159 leader_election.cc:288] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb [CANDIDATE]: Term 1 election: Requested vote from peers e507948f786346c1b434523d0eb94b99 (127.7.67.129:46053), a94be11c46444055b3afff63d2043a02 (127.7.67.130:44817)
I20260811 05:17:27.863884 2629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2ee9276a8724deeb0e85b9fa76350a5" candidate_uuid: "f38f4b41425747a59ed2a1807ccb79cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e507948f786346c1b434523d0eb94b99"
I20260811 05:17:27.863884 2856 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2ee9276a8724deeb0e85b9fa76350a5" candidate_uuid: "f38f4b41425747a59ed2a1807ccb79cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a94be11c46444055b3afff63d2043a02"
I20260811 05:17:27.864037 2629 raft_consensus.cc:3037] T e2ee9276a8724deeb0e85b9fa76350a5 P e507948f786346c1b434523d0eb94b99 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:27.864033 2856 raft_consensus.cc:3037] T e2ee9276a8724deeb0e85b9fa76350a5 P a94be11c46444055b3afff63d2043a02 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:27.865437 2629 raft_consensus.cc:2454] T e2ee9276a8724deeb0e85b9fa76350a5 P e507948f786346c1b434523d0eb94b99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f38f4b41425747a59ed2a1807ccb79cb in term 1.
I20260811 05:17:27.865437 2856 raft_consensus.cc:2454] T e2ee9276a8724deeb0e85b9fa76350a5 P a94be11c46444055b3afff63d2043a02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f38f4b41425747a59ed2a1807ccb79cb in term 1.
I20260811 05:17:27.865801 2959 leader_election.cc:302] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: a94be11c46444055b3afff63d2043a02, e507948f786346c1b434523d0eb94b99, f38f4b41425747a59ed2a1807ccb79cb; no voters:
I20260811 05:17:27.866039 3159 raft_consensus.cc:2781] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:27.866278 3159 raft_consensus.cc:686] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb [term 1 LEADER]: Becoming Leader. State: Replica: f38f4b41425747a59ed2a1807ccb79cb, State: Running, Role: LEADER
I20260811 05:17:27.866627 3159 consensus_queue.cc:237] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507948f786346c1b434523d0eb94b99" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46053 } } peers { permanent_uuid: "a94be11c46444055b3afff63d2043a02" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44817 } } peers { permanent_uuid: "f38f4b41425747a59ed2a1807ccb79cb" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 42267 } }
I20260811 05:17:27.870198 2411 catalog_manager.cc:5495] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb reported cstate change: term changed from 0 to 1, leader changed from <none> to f38f4b41425747a59ed2a1807ccb79cb (127.7.67.131). New cstate: current_term: 1 leader_uuid: "f38f4b41425747a59ed2a1807ccb79cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e507948f786346c1b434523d0eb94b99" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a94be11c46444055b3afff63d2043a02" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44817 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f38f4b41425747a59ed2a1807ccb79cb" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 42267 } health_report { overall_health: HEALTHY } } }
I20260811 05:17:27.889304 3147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:27.885545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35746 (local address 127.7.67.131:42267)
0811 05:17:27.885702 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:27.885707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:27.885723 (+ 16us) server_negotiation.cc:407] Connection header received
0811 05:17:27.885862 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:27.885865 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:27.885922 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:27.886040 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:17:27.886300 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.886817 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.887700 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:27.887942 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:27.888056 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:27.888387 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:17:27.888883 (+ 496us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:17:27.889017 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:27.889064 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 05:17:27.889121 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 05:17:27.895622 2856 raft_consensus.cc:1261] T e2ee9276a8724deeb0e85b9fa76350a5 P a94be11c46444055b3afff63d2043a02 [term 1 FOLLOWER]: Refusing update from remote peer f38f4b41425747a59ed2a1807ccb79cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:27.895656 2629 raft_consensus.cc:1261] T e2ee9276a8724deeb0e85b9fa76350a5 P e507948f786346c1b434523d0eb94b99 [term 1 FOLLOWER]: Refusing update from remote peer f38f4b41425747a59ed2a1807ccb79cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:27.896472 3166 consensus_queue.cc:1035] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "a94be11c46444055b3afff63d2043a02" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:27.896732 3159 consensus_queue.cc:1035] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e507948f786346c1b434523d0eb94b99" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 46053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:27.906824 3172 mvcc.cc:205] Tried to move back new op lower bound from 7317198634572365824 to 7317198634464464896. Current Snapshot: MvccSnapshot[applied={T|T < 7317198634572365824}]
I20260811 05:17:27.907491 3171 mvcc.cc:205] Tried to move back new op lower bound from 7317198634572365824 to 7317198634464464896. Current Snapshot: MvccSnapshot[applied={T|T < 7317198634572365824}]
I20260811 05:17:27.922410 2411 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:17:27.922619 2411 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:17:27.925518 2411 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9c358ad7e13543c6b6b08f8d4bf6e8ad: Sending DeleteTablet for 3 replicas of tablet e2ee9276a8724deeb0e85b9fa76350a5
I20260811 05:17:27.926465 3022 tablet_service.cc:1514] Processing DeleteTablet for tablet e2ee9276a8724deeb0e85b9fa76350a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:27 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:35732
I20260811 05:17:27.926465 2597 tablet_service.cc:1514] Processing DeleteTablet for tablet e2ee9276a8724deeb0e85b9fa76350a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:27 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:43252
I20260811 05:17:27.926502 2823 tablet_service.cc:1514] Processing DeleteTablet for tablet e2ee9276a8724deeb0e85b9fa76350a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:27 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:40786
I20260811 05:17:27.926990 3182 tablet_replica.cc:330] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb: stopping tablet replica
I20260811 05:17:27.927280 3185 tablet_replica.cc:330] T e2ee9276a8724deeb0e85b9fa76350a5 P e507948f786346c1b434523d0eb94b99: stopping tablet replica
I20260811 05:17:27.927451 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 2499
I20260811 05:17:27.927579 3185 raft_consensus.cc:2229] T e2ee9276a8724deeb0e85b9fa76350a5 P e507948f786346c1b434523d0eb94b99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:17:27.927793 3182 raft_consensus.cc:2229] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:17:27.927923 3185 raft_consensus.cc:2258] T e2ee9276a8724deeb0e85b9fa76350a5 P e507948f786346c1b434523d0eb94b99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:27.928194 3183 tablet_replica.cc:330] T e2ee9276a8724deeb0e85b9fa76350a5 P a94be11c46444055b3afff63d2043a02: stopping tablet replica
I20260811 05:17:27.928432 3182 raft_consensus.cc:2258] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:27.928491 3183 raft_consensus.cc:2229] T e2ee9276a8724deeb0e85b9fa76350a5 P a94be11c46444055b3afff63d2043a02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:17:27.928807 3183 raft_consensus.cc:2258] T e2ee9276a8724deeb0e85b9fa76350a5 P a94be11c46444055b3afff63d2043a02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:27.932245 3183 ts_tablet_manager.cc:1905] T e2ee9276a8724deeb0e85b9fa76350a5 P a94be11c46444055b3afff63d2043a02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:17:27.936390 3183 ts_tablet_manager.cc:1918] T e2ee9276a8724deeb0e85b9fa76350a5 P a94be11c46444055b3afff63d2043a02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:17:27.936584 3183 log.cc:1196] T e2ee9276a8724deeb0e85b9fa76350a5 P a94be11c46444055b3afff63d2043a02: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/e2ee9276a8724deeb0e85b9fa76350a5
I20260811 05:17:27.937054 3183 ts_tablet_manager.cc:1939] T e2ee9276a8724deeb0e85b9fa76350a5 P a94be11c46444055b3afff63d2043a02: Deleting consensus metadata
I20260811 05:17:27.938541 2392 catalog_manager.cc:4841] TS a94be11c46444055b3afff63d2043a02 (127.7.67.130:44817): tablet e2ee9276a8724deeb0e85b9fa76350a5 (table test-table [id=e8dd7c22a67f4c7886e74b8af408e6b5]) successfully deleted
I20260811 05:17:27.941856 3182 ts_tablet_manager.cc:1905] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 05:17:27.943174 2391 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.129:46053 (error 108)
I20260811 05:17:27.944113 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 2711
I20260811 05:17:27.944833 3141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:27.944471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:46053 (local address 127.0.0.1:43266)
0811 05:17:27.944652 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:27.944740 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.129:46053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 05:17:27.945528 3182 ts_tablet_manager.cc:1918] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:17:27.945624 3182 log.cc:1196] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/e2ee9276a8724deeb0e85b9fa76350a5
I20260811 05:17:27.945986 3182 ts_tablet_manager.cc:1939] T e2ee9276a8724deeb0e85b9fa76350a5 P f38f4b41425747a59ed2a1807ccb79cb: Deleting consensus metadata
W20260811 05:17:27.946744 2391 catalog_manager.cc:4584] TS e507948f786346c1b434523d0eb94b99 (127.7.67.129:46053): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e2ee9276a8724deeb0e85b9fa76350a5: Network error: Client connection negotiation failed: client connection to 127.7.67.129:46053: connect: Connection refused (error 111)
I20260811 05:17:27.947854 2390 catalog_manager.cc:4841] TS f38f4b41425747a59ed2a1807ccb79cb (127.7.67.131:42267): tablet e2ee9276a8724deeb0e85b9fa76350a5 (table test-table [id=e8dd7c22a67f4c7886e74b8af408e6b5]) successfully deleted
I20260811 05:17:27.958019 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 2931
I20260811 05:17:27.966741 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 2370
2026-08-11T05:17:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2619 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T05:17:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:28Z Disabled control of system clock
I20260811 05:17:28.011274 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:40645
--webserver_interface=127.7.67.190
--webserver_port=0
--builtin_ntp_servers=127.7.67.148:39691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:40645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 05:17:28.131212 3195 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:28.131496 3195 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:28.131534 3195 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:28.135409 3195 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:28.135469 3195 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:28.135488 3195 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:28.135509 3195 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:28.135526 3195 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:28.140305 3195 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:39691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:40645
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:40645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3195
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:28.141440 3195 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:28.142798 3195 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:28.151458 3207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:28.151432 3209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:28.151423 3206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:28.151677 3195 server_base.cc:1034] running on GCE node
I20260811 05:17:28.152444 3195 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:28.153561 3195 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:28.154752 3195 hybrid_clock.cc:648] HybridClock initialized: now 1786425448154720 us; error 44 us; skew 500 ppm
I20260811 05:17:28.156858 3195 webserver.cc:460] Webserver started at http://127.7.67.190:45153/ using document root <none> and password file <none>
I20260811 05:17:28.157495 3195 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:28.157564 3195 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:28.157820 3195 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:28.159970 3195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "1836c049374540c8a08eacb4adfcd011"
format_stamp: "Formatted at 2026-08-11 05:17:28 on dist-test-slave-04bb"
server_key: "1c3e3046f0be4ef5271d503183dd9ef8"
server_key_iv: "f8ddc293b2e2eb17523a713027d37767"
server_key_version: "encryptionkey@0"
I20260811 05:17:28.160651 3195 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "1836c049374540c8a08eacb4adfcd011"
format_stamp: "Formatted at 2026-08-11 05:17:28 on dist-test-slave-04bb"
server_key: "1c3e3046f0be4ef5271d503183dd9ef8"
server_key_iv: "f8ddc293b2e2eb17523a713027d37767"
server_key_version: "encryptionkey@0"
I20260811 05:17:28.164652 3195 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 05:17:28.167340 3215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:28.168763 3195 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:17:28.168913 3195 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "1836c049374540c8a08eacb4adfcd011"
format_stamp: "Formatted at 2026-08-11 05:17:28 on dist-test-slave-04bb"
server_key: "1c3e3046f0be4ef5271d503183dd9ef8"
server_key_iv: "f8ddc293b2e2eb17523a713027d37767"
server_key_version: "encryptionkey@0"
I20260811 05:17:28.169042 3195 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:28.184422 3195 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:28.185213 3195 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:28.185439 3195 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:28.193286 3195 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:40645
I20260811 05:17:28.193293 3296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:40645 every 8 connection(s)
I20260811 05:17:28.194324 3195 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:28.197788 3297 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:28.197992 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 3195
I20260811 05:17:28.198163 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:17:28.198482 7438 external_mini_cluster.cc:1428] Setting key 36141a6cda9464df0d377a1ba9f7b4d2
I20260811 05:17:28.205204 3297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1836c049374540c8a08eacb4adfcd011: Bootstrap starting.
I20260811 05:17:28.207526 3297 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1836c049374540c8a08eacb4adfcd011: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:28.208361 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:28.199928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57078 (local address 127.7.67.190:40645)
0811 05:17:28.200662 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:28.200681 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:28.200721 (+ 40us) server_negotiation.cc:407] Connection header received
0811 05:17:28.201427 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:28.201445 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:28.201855 (+ 410us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:28.202168 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:28.202526 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.203496 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.204748 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.205232 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.205438 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:28.206446 (+ 1008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:28.206452 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:28.206465 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:28.206871 (+ 406us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:28.206931 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:28.206953 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:28.207081 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:28.207218 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 05:17:28.207486 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":164,"threads_started":1}
I20260811 05:17:28.208443 3297 log.cc:824] T 00000000000000000000000000000000 P 1836c049374540c8a08eacb4adfcd011: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:28.210500 3297 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1836c049374540c8a08eacb4adfcd011: No bootstrap required, opened a new log
I20260811 05:17:28.214201 3297 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1836c049374540c8a08eacb4adfcd011 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1836c049374540c8a08eacb4adfcd011" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 40645 } }
I20260811 05:17:28.214432 3297 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1836c049374540c8a08eacb4adfcd011 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:28.214494 3297 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1836c049374540c8a08eacb4adfcd011 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1836c049374540c8a08eacb4adfcd011, State: Initialized, Role: FOLLOWER
I20260811 05:17:28.214915 3297 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1836c049374540c8a08eacb4adfcd011 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1836c049374540c8a08eacb4adfcd011" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 40645 } }
I20260811 05:17:28.215037 3297 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1836c049374540c8a08eacb4adfcd011 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:28.215106 3297 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1836c049374540c8a08eacb4adfcd011 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:28.215232 3297 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1836c049374540c8a08eacb4adfcd011 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:28.216260 3297 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1836c049374540c8a08eacb4adfcd011 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1836c049374540c8a08eacb4adfcd011" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 40645 } }
I20260811 05:17:28.216656 3297 leader_election.cc:302] T 00000000000000000000000000000000 P 1836c049374540c8a08eacb4adfcd011 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1836c049374540c8a08eacb4adfcd011; no voters:
I20260811 05:17:28.216991 3297 leader_election.cc:288] T 00000000000000000000000000000000 P 1836c049374540c8a08eacb4adfcd011 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:28.217131 3302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1836c049374540c8a08eacb4adfcd011 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:28.217383 3302 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1836c049374540c8a08eacb4adfcd011 [term 1 LEADER]: Becoming Leader. State: Replica: 1836c049374540c8a08eacb4adfcd011, State: Running, Role: LEADER
I20260811 05:17:28.217693 3302 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1836c049374540c8a08eacb4adfcd011 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1836c049374540c8a08eacb4adfcd011" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 40645 } }
I20260811 05:17:28.218266 3297 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1836c049374540c8a08eacb4adfcd011 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:28.219405 3303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1836c049374540c8a08eacb4adfcd011 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1836c049374540c8a08eacb4adfcd011" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1836c049374540c8a08eacb4adfcd011" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 40645 } } }
I20260811 05:17:28.219539 3303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1836c049374540c8a08eacb4adfcd011 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:28.219878 3305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1836c049374540c8a08eacb4adfcd011 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1836c049374540c8a08eacb4adfcd011. Latest consensus state: current_term: 1 leader_uuid: "1836c049374540c8a08eacb4adfcd011" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1836c049374540c8a08eacb4adfcd011" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 40645 } } }
I20260811 05:17:28.219985 3305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1836c049374540c8a08eacb4adfcd011 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:28.220075 3313 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:28.223243 3313 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:28.229236 3313 catalog_manager.cc:1347] Generated new cluster ID: 1132ad90a94f432a894d2cb27ea296b7
I20260811 05:17:28.229305 3313 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:28.255482 3313 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:28.256462 3313 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:28.266842 3313 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1836c049374540c8a08eacb4adfcd011: Generated new TSK 0
I20260811 05:17:28.267791 3313 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:28.275695 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:40645
--builtin_ntp_servers=127.7.67.148:39691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 05:17:28.392205 3327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:28.392583 3327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:28.392683 3327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:28.396692 3327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:28.396804 3327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:28.396906 3327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:17:28.401716 3327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:39691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:40645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:28.402889 3327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:28.404302 3327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:28.414541 3336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:28.414572 3333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:28.414635 3334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:28.414780 3327 server_base.cc:1034] running on GCE node
I20260811 05:17:28.415318 3327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:28.416064 3327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:28.417255 3327 hybrid_clock.cc:648] HybridClock initialized: now 1786425448417241 us; error 29 us; skew 500 ppm
I20260811 05:17:28.419276 3327 webserver.cc:460] Webserver started at http://127.7.67.129:42113/ using document root <none> and password file <none>
I20260811 05:17:28.419962 3327 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:28.420059 3327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:28.420289 3327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:28.422443 3327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "77f40239b3824ab296b54f19b2ff74c0"
format_stamp: "Formatted at 2026-08-11 05:17:28 on dist-test-slave-04bb"
server_key: "3851475748d22516a4d81c28d3a76a8a"
server_key_iv: "7520f3be9555bb6b07be46194c3f5f4e"
server_key_version: "encryptionkey@0"
I20260811 05:17:28.423048 3327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "77f40239b3824ab296b54f19b2ff74c0"
format_stamp: "Formatted at 2026-08-11 05:17:28 on dist-test-slave-04bb"
server_key: "3851475748d22516a4d81c28d3a76a8a"
server_key_iv: "7520f3be9555bb6b07be46194c3f5f4e"
server_key_version: "encryptionkey@0"
I20260811 05:17:28.426896 3327 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 05:17:28.429394 3342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:28.430586 3327 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260811 05:17:28.430725 3327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "77f40239b3824ab296b54f19b2ff74c0"
format_stamp: "Formatted at 2026-08-11 05:17:28 on dist-test-slave-04bb"
server_key: "3851475748d22516a4d81c28d3a76a8a"
server_key_iv: "7520f3be9555bb6b07be46194c3f5f4e"
server_key_version: "encryptionkey@0"
I20260811 05:17:28.430845 3327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:28.441946 3327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:28.442576 3327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:28.442778 3327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:28.443387 3327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:28.444326 3327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:28.444399 3327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:28.444473 3327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:28.444516 3327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:28.453614 3327 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:45117
I20260811 05:17:28.453650 3512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:45117 every 8 connection(s)
I20260811 05:17:28.454520 3327 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:17:28.463423 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:28.456411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:40123 (local address 127.7.67.190:40645)
0811 05:17:28.456608 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:28.456612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:28.457662 (+ 1050us) server_negotiation.cc:407] Connection header received
0811 05:17:28.459047 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:28.459050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:28.459097 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:28.459170 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:28.460047 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.460622 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.461713 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.461914 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.461987 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:28.463000 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:28.463003 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:28.463005 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:28.463016 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:28.463033 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:28.463041 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:28.463101 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:28.463251 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 05:17:28.463309 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 05:17:28.463660 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 3327
I20260811 05:17:28.463778 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:17:28.464107 7438 external_mini_cluster.cc:1428] Setting key 127b6d7d62f80f3c8ef23602f98d40a0
I20260811 05:17:28.464505 3515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:28.456887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:40645 (local address 127.7.67.129:40123)
0811 05:17:28.457535 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:28.457575 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:28.458853 (+ 1278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:28.459339 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:28.459346 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:28.459427 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:28.459913 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:28.459925 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.460735 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.460738 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:28.461603 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:28.461612 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.462043 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.462045 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:28.462184 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:28.462747 (+ 563us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:17:28.462748 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:28.462788 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:17:28.462855 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:28.463121 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:28.463128 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:28.463132 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:17:28.463228 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 05:17:28.463502 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":117,"threads_started":1}
I20260811 05:17:28.465859 3513 heartbeater.cc:347] Connected to a master server at 127.7.67.190:40645
I20260811 05:17:28.466089 3513 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:28.466392 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:40645
--builtin_ntp_servers=127.7.67.148:39691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 05:17:28.466682 3513 heartbeater.cc:511] Master 127.7.67.190:40645 requested a full tablet report, sending...
I20260811 05:17:28.468428 3238 ts_manager.cc:194] Registered new tserver with Master: 77f40239b3824ab296b54f19b2ff74c0 (127.7.67.129:45117)
I20260811 05:17:28.470129 3238 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.67.129:40123
W20260811 05:17:28.580593 3517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:28.580889 3517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:28.580960 3517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:28.584885 3517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:28.584959 3517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:28.585050 3517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:17:28.589653 3517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:39691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:40645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:28.590698 3517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:28.592065 3517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:17:28.601672 3517 server_base.cc:1034] running on GCE node
W20260811 05:17:28.601605 3524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:28.601622 3527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:28.601624 3523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:28.602252 3517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:28.602896 3517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:28.604080 3517 hybrid_clock.cc:648] HybridClock initialized: now 1786425448604050 us; error 52 us; skew 500 ppm
I20260811 05:17:28.605938 3517 webserver.cc:460] Webserver started at http://127.7.67.130:35723/ using document root <none> and password file <none>
I20260811 05:17:28.606510 3517 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:28.606603 3517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:28.606861 3517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:28.609040 3517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "a440dc1946004e438c3cc04e0776c55a"
format_stamp: "Formatted at 2026-08-11 05:17:28 on dist-test-slave-04bb"
server_key: "23a37a31e56f85ccfd5f5575f08bc94c"
server_key_iv: "158369954ca8b6cb87b43a11f401dc47"
server_key_version: "encryptionkey@0"
I20260811 05:17:28.609606 3517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "a440dc1946004e438c3cc04e0776c55a"
format_stamp: "Formatted at 2026-08-11 05:17:28 on dist-test-slave-04bb"
server_key: "23a37a31e56f85ccfd5f5575f08bc94c"
server_key_iv: "158369954ca8b6cb87b43a11f401dc47"
server_key_version: "encryptionkey@0"
I20260811 05:17:28.613507 3517 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 05:17:28.616460 3539 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:28.617704 3517 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:17:28.617846 3517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "a440dc1946004e438c3cc04e0776c55a"
format_stamp: "Formatted at 2026-08-11 05:17:28 on dist-test-slave-04bb"
server_key: "23a37a31e56f85ccfd5f5575f08bc94c"
server_key_iv: "158369954ca8b6cb87b43a11f401dc47"
server_key_version: "encryptionkey@0"
I20260811 05:17:28.617959 3517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:28.629822 3517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:28.630367 3517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:28.630514 3517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:28.631001 3517 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:28.631839 3517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:28.631906 3517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:28.631942 3517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:28.631963 3517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:28.641839 3517 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:44825
I20260811 05:17:28.641865 3713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:44825 every 8 connection(s)
I20260811 05:17:28.642700 3517 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:17:28.644168 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 3517
I20260811 05:17:28.644304 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:17:28.644685 7438 external_mini_cluster.cc:1428] Setting key 0989501bcf45afe6d7757f5fdaa1e366
I20260811 05:17:28.646978 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:40645
--builtin_ntp_servers=127.7.67.148:39691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 05:17:28.653252 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:28.644643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:40021 (local address 127.7.67.190:40645)
0811 05:17:28.644809 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:28.644814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:28.646019 (+ 1205us) server_negotiation.cc:407] Connection header received
0811 05:17:28.647913 (+ 1894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:28.647916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:28.647963 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:28.648036 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:28.649269 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.649823 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.650994 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.651223 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.651341 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:28.652716 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:28.652720 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:28.652729 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:28.652742 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:28.652763 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:28.652771 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:28.652859 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:28.653037 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 05:17:28.653103 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 05:17:28.654114 3718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:28.645108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:40645 (local address 127.7.67.130:40021)
0811 05:17:28.645841 (+ 733us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:28.645896 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:28.647641 (+ 1745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:28.648219 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:28.648230 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:28.648352 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:28.649106 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:28.649125 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.649957 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.649960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:28.650852 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:28.650863 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.651384 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.651388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:28.651592 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:28.652397 (+ 805us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:17:28.652399 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:28.652442 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:17:28.652507 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:28.652862 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:28.652874 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:28.652881 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 05:17:28.653021 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 05:17:28.653323 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":109,"threads_started":1}
I20260811 05:17:28.655344 3714 heartbeater.cc:347] Connected to a master server at 127.7.67.190:40645
I20260811 05:17:28.655628 3714 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:28.656183 3714 heartbeater.cc:511] Master 127.7.67.190:40645 requested a full tablet report, sending...
I20260811 05:17:28.657393 3238 ts_manager.cc:194] Registered new tserver with Master: a440dc1946004e438c3cc04e0776c55a (127.7.67.130:44825)
I20260811 05:17:28.658299 3238 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.67.130:40021
W20260811 05:17:28.761969 3719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:28.762212 3719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:28.762248 3719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:28.766109 3719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:28.766152 3719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:28.766203 3719 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:17:28.771136 3719 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:39691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:40645
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:28.772166 3719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:28.773670 3719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:28.783227 3726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:28.783277 3730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:28.783495 3719 server_base.cc:1034] running on GCE node
W20260811 05:17:28.783231 3727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:28.784094 3719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:28.784811 3719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:28.785986 3719 hybrid_clock.cc:648] HybridClock initialized: now 1786425448785971 us; error 24 us; skew 500 ppm
I20260811 05:17:28.787920 3719 webserver.cc:460] Webserver started at http://127.7.67.131:34015/ using document root <none> and password file <none>
I20260811 05:17:28.788697 3719 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:28.788775 3719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:28.789007 3719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:28.791152 3719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "b8c68b27658f4220bd0b15f51635197f"
format_stamp: "Formatted at 2026-08-11 05:17:28 on dist-test-slave-04bb"
server_key: "0707b421c043aa84635ad6670d16fc44"
server_key_iv: "0c0a865504862e8e34939ac7f850bd73"
server_key_version: "encryptionkey@0"
I20260811 05:17:28.791756 3719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "b8c68b27658f4220bd0b15f51635197f"
format_stamp: "Formatted at 2026-08-11 05:17:28 on dist-test-slave-04bb"
server_key: "0707b421c043aa84635ad6670d16fc44"
server_key_iv: "0c0a865504862e8e34939ac7f850bd73"
server_key_version: "encryptionkey@0"
I20260811 05:17:28.795599 3719 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 05:17:28.798359 3737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:28.799551 3719 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:17:28.799695 3719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "b8c68b27658f4220bd0b15f51635197f"
format_stamp: "Formatted at 2026-08-11 05:17:28 on dist-test-slave-04bb"
server_key: "0707b421c043aa84635ad6670d16fc44"
server_key_iv: "0c0a865504862e8e34939ac7f850bd73"
server_key_version: "encryptionkey@0"
I20260811 05:17:28.799810 3719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:28.828456 3719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:28.829282 3719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:28.829506 3719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:28.830159 3719 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:28.831203 3719 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:28.831279 3719 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:28.831355 3719 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:28.831400 3719 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:28.840919 3719 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:38141
I20260811 05:17:28.840958 3906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:38141 every 8 connection(s)
I20260811 05:17:28.841872 3719 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:17:28.844521 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 3719
I20260811 05:17:28.844650 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:17:28.844971 7438 external_mini_cluster.cc:1428] Setting key 2d2d9e0bea6980ae4970fc4d273cd66e
I20260811 05:17:28.851776 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:28.843764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:33341 (local address 127.7.67.190:40645)
0811 05:17:28.843909 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:28.843919 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:28.845150 (+ 1231us) server_negotiation.cc:407] Connection header received
0811 05:17:28.846962 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:28.846966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:28.847022 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:28.847114 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:28.848159 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.848777 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.849880 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.850098 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.850184 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:28.851349 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:28.851352 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:28.851354 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:28.851366 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:28.851385 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:28.851393 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:28.851459 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:28.851607 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 05:17:28.851659 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 05:17:28.852700 3910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:28.844132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:40645 (local address 127.7.67.131:33341)
0811 05:17:28.844980 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:28.845041 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:28.846739 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:28.847343 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:28.847350 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:28.847477 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:28.848005 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:28.848020 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.848891 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.848893 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:28.849764 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:28.849774 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.850236 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.850239 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:28.850393 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:28.851044 (+ 651us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:17:28.851045 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:28.851086 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:17:28.851148 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:28.851474 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:28.851480 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:28.851484 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:17:28.851578 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 05:17:28.851796 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":118,"threads_started":1}
I20260811 05:17:28.853920 3907 heartbeater.cc:347] Connected to a master server at 127.7.67.190:40645
I20260811 05:17:28.854158 3907 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:28.854689 3907 heartbeater.cc:511] Master 127.7.67.190:40645 requested a full tablet report, sending...
I20260811 05:17:28.855751 3238 ts_manager.cc:194] Registered new tserver with Master: b8c68b27658f4220bd0b15f51635197f (127.7.67.131:38141)
I20260811 05:17:28.856320 3238 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.67.131:33341
I20260811 05:17:28.859359 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:17:28.864890 3300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:28.861355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57086 (local address 127.7.67.190:40645)
0811 05:17:28.861506 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:28.861511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:28.861687 (+ 176us) server_negotiation.cc:407] Connection header received
0811 05:17:28.861776 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:28.861778 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:28.861818 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:28.861876 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:28.862132 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.862625 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.863391 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.863595 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.863672 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:28.864390 (+ 718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:28.864393 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:28.864395 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:28.864406 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:28.864422 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:28.864429 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:28.864487 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:28.864644 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 05:17:28.864693 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 05:17:28.868116 3238 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:17:28.871263 3238 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:17:28.886809 3926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:28.880837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:38141 (local address 127.0.0.1:60570)
0811 05:17:28.881621 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:28.881633 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:28.881707 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:28.882579 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:28.882582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:28.882614 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:28.882879 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:28.882887 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.883958 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.883965 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:28.885343 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:28.885354 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.886260 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.886263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:28.886369 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:28.886407 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 05:17:28.886515 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 05:17:28.886636 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"mutex_wait_us":94,"spinlock_wait_cycles":6528,"thread_start_us":103,"threads_started":1}
I20260811 05:17:28.886811 3931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:28.881344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60570 (local address 127.7.67.131:38141)
0811 05:17:28.882001 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:28.882016 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:28.882042 (+ 26us) server_negotiation.cc:407] Connection header received
0811 05:17:28.882125 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:28.882130 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:28.882420 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:28.882738 (+ 318us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:28.883041 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.883835 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.885482 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.886124 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.886259 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:28.886478 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:28.886540 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 05:17:28.886637 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":158,"threads_started":1}
I20260811 05:17:28.887244 3921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:28.880619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:44825 (local address 127.0.0.1:45630)
0811 05:17:28.881472 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:28.881526 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:28.881670 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:28.882727 (+ 1057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:28.882730 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:28.882752 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:28.883062 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:28.883069 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.884405 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.884411 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:28.885530 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:28.885540 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.886794 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.886798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:28.886922 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:28.886957 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 05:17:28.887038 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 05:17:28.887101 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"mutex_wait_us":32,"thread_start_us":105,"threads_started":1}
I20260811 05:17:28.887710 3925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:28.881081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:45117 (local address 127.0.0.1:43798)
0811 05:17:28.881919 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:28.881932 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:28.882009 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:28.882571 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:28.882577 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:28.882616 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:28.882828 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:28.882837 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.884340 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.884344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:28.885493 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:28.885501 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.886794 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.886798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:28.887413 (+ 615us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:28.887440 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 05:17:28.887511 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 05:17:28.887573 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"mutex_wait_us":430,"spinlock_wait_cycles":9216,"thread_start_us":106,"threads_started":1}
I20260811 05:17:28.887715 3923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:28.880808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45630 (local address 127.7.67.130:44825)
0811 05:17:28.882267 (+ 1459us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:28.882279 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:28.882301 (+ 22us) server_negotiation.cc:407] Connection header received
0811 05:17:28.882371 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:28.882375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:28.882534 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:28.882855 (+ 321us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:28.883192 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.884316 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.885655 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.886433 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.886544 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:28.886723 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:28.887441 (+ 718us) server_negotiation.cc:299] Negotiation successful
0811 05:17:28.887550 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1344,"thread_start_us":117,"threads_started":1}
I20260811 05:17:28.888291 3933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:28.881533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43798 (local address 127.7.67.129:45117)
0811 05:17:28.882005 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:28.882017 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:28.882044 (+ 27us) server_negotiation.cc:407] Connection header received
0811 05:17:28.882131 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:28.882137 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:28.882356 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:28.882508 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:28.882944 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.884188 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.885618 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:28.886433 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:28.886543 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:28.886736 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:28.888027 (+ 1291us) server_negotiation.cc:299] Negotiation successful
0811 05:17:28.888131 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":138,"threads_started":1}
I20260811 05:17:28.890250 3810 tablet_service.cc:1467] Processing CreateTablet for tablet 4ec704647e92449a8e0520a298865f29 (DEFAULT_TABLE table=test-table [id=2be60d8b168144c58ba0989622222886]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:28.890604 3611 tablet_service.cc:1467] Processing CreateTablet for tablet 4ec704647e92449a8e0520a298865f29 (DEFAULT_TABLE table=test-table [id=2be60d8b168144c58ba0989622222886]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:28.890793 3417 tablet_service.cc:1467] Processing CreateTablet for tablet 4ec704647e92449a8e0520a298865f29 (DEFAULT_TABLE table=test-table [id=2be60d8b168144c58ba0989622222886]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:28.891384 3810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ec704647e92449a8e0520a298865f29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:28.891706 3611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ec704647e92449a8e0520a298865f29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:28.891706 3417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ec704647e92449a8e0520a298865f29. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:28.898303 3934 tablet_bootstrap.cc:492] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f: Bootstrap starting.
I20260811 05:17:28.899771 3936 tablet_bootstrap.cc:492] T 4ec704647e92449a8e0520a298865f29 P a440dc1946004e438c3cc04e0776c55a: Bootstrap starting.
I20260811 05:17:28.899771 3935 tablet_bootstrap.cc:492] T 4ec704647e92449a8e0520a298865f29 P 77f40239b3824ab296b54f19b2ff74c0: Bootstrap starting.
I20260811 05:17:28.900679 3934 tablet_bootstrap.cc:654] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:28.901438 3934 log.cc:824] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:28.902029 3936 tablet_bootstrap.cc:654] T 4ec704647e92449a8e0520a298865f29 P a440dc1946004e438c3cc04e0776c55a: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:28.902029 3935 tablet_bootstrap.cc:654] T 4ec704647e92449a8e0520a298865f29 P 77f40239b3824ab296b54f19b2ff74c0: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:28.902887 3935 log.cc:824] T 4ec704647e92449a8e0520a298865f29 P 77f40239b3824ab296b54f19b2ff74c0: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:28.902887 3936 log.cc:824] T 4ec704647e92449a8e0520a298865f29 P a440dc1946004e438c3cc04e0776c55a: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:28.903587 3934 tablet_bootstrap.cc:492] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f: No bootstrap required, opened a new log
I20260811 05:17:28.903800 3934 ts_tablet_manager.cc:1397] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.003s
I20260811 05:17:28.904922 3936 tablet_bootstrap.cc:492] T 4ec704647e92449a8e0520a298865f29 P a440dc1946004e438c3cc04e0776c55a: No bootstrap required, opened a new log
I20260811 05:17:28.904927 3935 tablet_bootstrap.cc:492] T 4ec704647e92449a8e0520a298865f29 P 77f40239b3824ab296b54f19b2ff74c0: No bootstrap required, opened a new log
I20260811 05:17:28.905113 3935 ts_tablet_manager.cc:1397] T 4ec704647e92449a8e0520a298865f29 P 77f40239b3824ab296b54f19b2ff74c0: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260811 05:17:28.905113 3936 ts_tablet_manager.cc:1397] T 4ec704647e92449a8e0520a298865f29 P a440dc1946004e438c3cc04e0776c55a: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 05:17:28.907248 3934 raft_consensus.cc:348] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a440dc1946004e438c3cc04e0776c55a" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44825 } } peers { permanent_uuid: "b8c68b27658f4220bd0b15f51635197f" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38141 } } peers { permanent_uuid: "77f40239b3824ab296b54f19b2ff74c0" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 45117 } }
I20260811 05:17:28.907485 3934 raft_consensus.cc:374] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:28.907538 3934 raft_consensus.cc:729] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c68b27658f4220bd0b15f51635197f, State: Initialized, Role: FOLLOWER
I20260811 05:17:28.908030 3934 consensus_queue.cc:260] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a440dc1946004e438c3cc04e0776c55a" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44825 } } peers { permanent_uuid: "b8c68b27658f4220bd0b15f51635197f" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38141 } } peers { permanent_uuid: "77f40239b3824ab296b54f19b2ff74c0" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 45117 } }
I20260811 05:17:28.908721 3907 heartbeater.cc:503] Master 127.7.67.190:40645 was elected leader, sending a full tablet report...
I20260811 05:17:28.909085 3934 ts_tablet_manager.cc:1428] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f: Time spent starting tablet: real 0.005s user 0.005s sys 0.002s
I20260811 05:17:28.909415 3935 raft_consensus.cc:348] T 4ec704647e92449a8e0520a298865f29 P 77f40239b3824ab296b54f19b2ff74c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a440dc1946004e438c3cc04e0776c55a" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44825 } } peers { permanent_uuid: "b8c68b27658f4220bd0b15f51635197f" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38141 } } peers { permanent_uuid: "77f40239b3824ab296b54f19b2ff74c0" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 45117 } }
I20260811 05:17:28.909436 3936 raft_consensus.cc:348] T 4ec704647e92449a8e0520a298865f29 P a440dc1946004e438c3cc04e0776c55a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a440dc1946004e438c3cc04e0776c55a" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44825 } } peers { permanent_uuid: "b8c68b27658f4220bd0b15f51635197f" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38141 } } peers { permanent_uuid: "77f40239b3824ab296b54f19b2ff74c0" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 45117 } }
I20260811 05:17:28.909662 3936 raft_consensus.cc:374] T 4ec704647e92449a8e0520a298865f29 P a440dc1946004e438c3cc04e0776c55a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:28.909662 3935 raft_consensus.cc:374] T 4ec704647e92449a8e0520a298865f29 P 77f40239b3824ab296b54f19b2ff74c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:28.909719 3936 raft_consensus.cc:729] T 4ec704647e92449a8e0520a298865f29 P a440dc1946004e438c3cc04e0776c55a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a440dc1946004e438c3cc04e0776c55a, State: Initialized, Role: FOLLOWER
I20260811 05:17:28.909730 3935 raft_consensus.cc:729] T 4ec704647e92449a8e0520a298865f29 P 77f40239b3824ab296b54f19b2ff74c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77f40239b3824ab296b54f19b2ff74c0, State: Initialized, Role: FOLLOWER
I20260811 05:17:28.910161 3935 consensus_queue.cc:260] T 4ec704647e92449a8e0520a298865f29 P 77f40239b3824ab296b54f19b2ff74c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a440dc1946004e438c3cc04e0776c55a" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44825 } } peers { permanent_uuid: "b8c68b27658f4220bd0b15f51635197f" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38141 } } peers { permanent_uuid: "77f40239b3824ab296b54f19b2ff74c0" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 45117 } }
I20260811 05:17:28.910161 3936 consensus_queue.cc:260] T 4ec704647e92449a8e0520a298865f29 P a440dc1946004e438c3cc04e0776c55a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a440dc1946004e438c3cc04e0776c55a" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44825 } } peers { permanent_uuid: "b8c68b27658f4220bd0b15f51635197f" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38141 } } peers { permanent_uuid: "77f40239b3824ab296b54f19b2ff74c0" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 45117 } }
I20260811 05:17:28.911118 3936 ts_tablet_manager.cc:1428] T 4ec704647e92449a8e0520a298865f29 P a440dc1946004e438c3cc04e0776c55a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 05:17:28.911118 3935 ts_tablet_manager.cc:1428] T 4ec704647e92449a8e0520a298865f29 P 77f40239b3824ab296b54f19b2ff74c0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 05:17:28.911473 3513 heartbeater.cc:503] Master 127.7.67.190:40645 was elected leader, sending a full tablet report...
I20260811 05:17:28.911482 3714 heartbeater.cc:503] Master 127.7.67.190:40645 was elected leader, sending a full tablet report...
W20260811 05:17:28.955772 3514 tablet.cc:2369] T 4ec704647e92449a8e0520a298865f29 P 77f40239b3824ab296b54f19b2ff74c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:17:29.092905 3908 tablet.cc:2369] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:17:29.143846 3717 tablet.cc:2369] T 4ec704647e92449a8e0520a298865f29 P a440dc1946004e438c3cc04e0776c55a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:17:29.145969 3942 raft_consensus.cc:482] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:17:29.146185 3942 raft_consensus.cc:504] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a440dc1946004e438c3cc04e0776c55a" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44825 } } peers { permanent_uuid: "b8c68b27658f4220bd0b15f51635197f" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38141 } } peers { permanent_uuid: "77f40239b3824ab296b54f19b2ff74c0" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 45117 } }
I20260811 05:17:29.147411 3942 leader_election.cc:288] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a440dc1946004e438c3cc04e0776c55a (127.7.67.130:44825), 77f40239b3824ab296b54f19b2ff74c0 (127.7.67.129:45117)
I20260811 05:17:29.151288 3910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:29.147700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:45117 (local address 127.7.67.131:40977)
0811 05:17:29.147939 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:29.147956 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:29.148069 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:29.148359 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:29.148362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:29.148379 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:29.148501 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:29.148507 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:29.149374 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:29.149380 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:29.150237 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:29.150246 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:29.150863 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:29.150867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:29.150964 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:29.150992 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 05:17:29.151081 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 05:17:29.151151 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":40}
I20260811 05:17:29.151292 3933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:29.147826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:40977 (local address 127.7.67.129:45117)
0811 05:17:29.147967 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:29.147974 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:29.148004 (+ 30us) server_negotiation.cc:407] Connection header received
0811 05:17:29.148204 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:29.148207 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:29.148252 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:29.148345 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:29.148640 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:29.149257 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:29.150367 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:29.150740 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:29.150867 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:29.150918 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:29.151100 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 05:17:29.151165 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 05:17:29.151484 3923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:29.147786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:56645 (local address 127.7.67.130:44825)
0811 05:17:29.147958 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:29.147963 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:29.148280 (+ 317us) server_negotiation.cc:407] Connection header received
0811 05:17:29.148441 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:29.148444 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:29.148495 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:29.148617 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:29.148917 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:29.149404 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:29.150485 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:29.150886 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:29.151023 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:29.151070 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:29.151262 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 05:17:29.151333 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 05:17:29.151662 3947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:29.147720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:44825 (local address 127.7.67.131:56645)
0811 05:17:29.148141 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:29.148153 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:29.148317 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:29.148634 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:29.148637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:29.148654 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:29.148784 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:29.148790 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:29.149524 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:29.149528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:29.150352 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:29.150361 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:29.151023 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:29.151027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:29.151129 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:29.151147 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 05:17:29.151244 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 05:17:29.151310 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":27,"thread_start_us":146,"threads_started":1}
I20260811 05:17:29.151943 3447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ec704647e92449a8e0520a298865f29" candidate_uuid: "b8c68b27658f4220bd0b15f51635197f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77f40239b3824ab296b54f19b2ff74c0" is_pre_election: true
I20260811 05:17:29.152263 3447 raft_consensus.cc:2454] T 4ec704647e92449a8e0520a298865f29 P 77f40239b3824ab296b54f19b2ff74c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c68b27658f4220bd0b15f51635197f in term 0.
I20260811 05:17:29.152380 3646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ec704647e92449a8e0520a298865f29" candidate_uuid: "b8c68b27658f4220bd0b15f51635197f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a440dc1946004e438c3cc04e0776c55a" is_pre_election: true
I20260811 05:17:29.152695 3646 raft_consensus.cc:2454] T 4ec704647e92449a8e0520a298865f29 P a440dc1946004e438c3cc04e0776c55a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c68b27658f4220bd0b15f51635197f in term 0.
I20260811 05:17:29.152817 3738 leader_election.cc:302] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77f40239b3824ab296b54f19b2ff74c0, b8c68b27658f4220bd0b15f51635197f; no voters:
I20260811 05:17:29.153139 3942 raft_consensus.cc:2781] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:17:29.153275 3942 raft_consensus.cc:482] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:17:29.153354 3942 raft_consensus.cc:3037] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:29.154353 3942 raft_consensus.cc:504] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a440dc1946004e438c3cc04e0776c55a" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44825 } } peers { permanent_uuid: "b8c68b27658f4220bd0b15f51635197f" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38141 } } peers { permanent_uuid: "77f40239b3824ab296b54f19b2ff74c0" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 45117 } }
I20260811 05:17:29.154676 3942 leader_election.cc:288] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f [CANDIDATE]: Term 1 election: Requested vote from peers a440dc1946004e438c3cc04e0776c55a (127.7.67.130:44825), 77f40239b3824ab296b54f19b2ff74c0 (127.7.67.129:45117)
I20260811 05:17:29.155031 3646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ec704647e92449a8e0520a298865f29" candidate_uuid: "b8c68b27658f4220bd0b15f51635197f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a440dc1946004e438c3cc04e0776c55a"
I20260811 05:17:29.155031 3447 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ec704647e92449a8e0520a298865f29" candidate_uuid: "b8c68b27658f4220bd0b15f51635197f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "77f40239b3824ab296b54f19b2ff74c0"
I20260811 05:17:29.155150 3646 raft_consensus.cc:3037] T 4ec704647e92449a8e0520a298865f29 P a440dc1946004e438c3cc04e0776c55a [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:29.155150 3447 raft_consensus.cc:3037] T 4ec704647e92449a8e0520a298865f29 P 77f40239b3824ab296b54f19b2ff74c0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:29.156416 3646 raft_consensus.cc:2454] T 4ec704647e92449a8e0520a298865f29 P a440dc1946004e438c3cc04e0776c55a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c68b27658f4220bd0b15f51635197f in term 1.
I20260811 05:17:29.156416 3447 raft_consensus.cc:2454] T 4ec704647e92449a8e0520a298865f29 P 77f40239b3824ab296b54f19b2ff74c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c68b27658f4220bd0b15f51635197f in term 1.
I20260811 05:17:29.156759 3738 leader_election.cc:302] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 77f40239b3824ab296b54f19b2ff74c0, b8c68b27658f4220bd0b15f51635197f; no voters:
I20260811 05:17:29.156922 3942 raft_consensus.cc:2781] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:29.157133 3942 raft_consensus.cc:686] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f [term 1 LEADER]: Becoming Leader. State: Replica: b8c68b27658f4220bd0b15f51635197f, State: Running, Role: LEADER
I20260811 05:17:29.157467 3942 consensus_queue.cc:237] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a440dc1946004e438c3cc04e0776c55a" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44825 } } peers { permanent_uuid: "b8c68b27658f4220bd0b15f51635197f" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38141 } } peers { permanent_uuid: "77f40239b3824ab296b54f19b2ff74c0" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 45117 } }
I20260811 05:17:29.160771 3238 catalog_manager.cc:5495] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f reported cstate change: term changed from 0 to 1, leader changed from <none> to b8c68b27658f4220bd0b15f51635197f (127.7.67.131). New cstate: current_term: 1 leader_uuid: "b8c68b27658f4220bd0b15f51635197f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a440dc1946004e438c3cc04e0776c55a" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44825 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8c68b27658f4220bd0b15f51635197f" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38141 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "77f40239b3824ab296b54f19b2ff74c0" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 45117 } health_report { overall_health: UNKNOWN } } }
I20260811 05:17:29.179462 3931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:29.175886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60574 (local address 127.7.67.131:38141)
0811 05:17:29.176031 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:29.176036 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:29.176050 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:17:29.176159 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:29.176162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:29.176214 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:29.176332 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:17:29.176632 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:29.177131 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:29.178008 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:29.178224 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:29.178337 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:29.178649 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:17:29.179054 (+ 405us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:17:29.179186 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:29.179251 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 05:17:29.179301 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 05:17:29.185464 3447 raft_consensus.cc:1261] T 4ec704647e92449a8e0520a298865f29 P 77f40239b3824ab296b54f19b2ff74c0 [term 1 FOLLOWER]: Refusing update from remote peer b8c68b27658f4220bd0b15f51635197f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:29.185462 3646 raft_consensus.cc:1261] T 4ec704647e92449a8e0520a298865f29 P a440dc1946004e438c3cc04e0776c55a [term 1 FOLLOWER]: Refusing update from remote peer b8c68b27658f4220bd0b15f51635197f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:29.186208 3942 consensus_queue.cc:1035] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f [LEADER]: Connected to new peer: Peer: permanent_uuid: "77f40239b3824ab296b54f19b2ff74c0" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 45117 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:29.186425 3948 consensus_queue.cc:1035] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a440dc1946004e438c3cc04e0776c55a" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 44825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:29.193234 3952 mvcc.cc:205] Tried to move back new op lower bound from 7317198639856107520 to 7317198639751491584. Current Snapshot: MvccSnapshot[applied={T|T < 7317198639856107520}]
I20260811 05:17:29.196853 3954 mvcc.cc:205] Tried to move back new op lower bound from 7317198639856107520 to 7317198639751491584. Current Snapshot: MvccSnapshot[applied={T|T < 7317198639856107520}]
I20260811 05:17:29.206215 3238 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:17:29.206382 3238 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:17:29.208791 3238 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1836c049374540c8a08eacb4adfcd011: Sending DeleteTablet for 3 replicas of tablet 4ec704647e92449a8e0520a298865f29
I20260811 05:17:29.209556 3611 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ec704647e92449a8e0520a298865f29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:29 UTC) from {username='slave'} at 127.0.0.1:45630
I20260811 05:17:29.209969 3417 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ec704647e92449a8e0520a298865f29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:29 UTC) from {username='slave'} at 127.0.0.1:43798
I20260811 05:17:29.210187 3971 tablet_replica.cc:330] T 4ec704647e92449a8e0520a298865f29 P a440dc1946004e438c3cc04e0776c55a: stopping tablet replica
I20260811 05:17:29.210440 3972 tablet_replica.cc:330] T 4ec704647e92449a8e0520a298865f29 P 77f40239b3824ab296b54f19b2ff74c0: stopping tablet replica
I20260811 05:17:29.210505 3971 raft_consensus.cc:2229] T 4ec704647e92449a8e0520a298865f29 P a440dc1946004e438c3cc04e0776c55a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:17:29.210733 3972 raft_consensus.cc:2229] T 4ec704647e92449a8e0520a298865f29 P 77f40239b3824ab296b54f19b2ff74c0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:17:29.210843 3971 raft_consensus.cc:2258] T 4ec704647e92449a8e0520a298865f29 P a440dc1946004e438c3cc04e0776c55a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:29.211158 3972 raft_consensus.cc:2258] T 4ec704647e92449a8e0520a298865f29 P 77f40239b3824ab296b54f19b2ff74c0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:29.211810 3810 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ec704647e92449a8e0520a298865f29 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:29 UTC) from {username='slave'} at 127.0.0.1:60570
I20260811 05:17:29.212304 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 3327
I20260811 05:17:29.220908 3974 tablet_replica.cc:330] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f: stopping tablet replica
I20260811 05:17:29.221365 3974 raft_consensus.cc:2229] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:17:29.221880 3974 raft_consensus.cc:2258] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:29.222466 3971 ts_tablet_manager.cc:1905] T 4ec704647e92449a8e0520a298865f29 P a440dc1946004e438c3cc04e0776c55a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:17:29.224066 3974 ts_tablet_manager.cc:1905] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:17:29.225504 3971 ts_tablet_manager.cc:1918] T 4ec704647e92449a8e0520a298865f29 P a440dc1946004e438c3cc04e0776c55a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:17:29.225610 3971 log.cc:1196] T 4ec704647e92449a8e0520a298865f29 P a440dc1946004e438c3cc04e0776c55a: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/4ec704647e92449a8e0520a298865f29
I20260811 05:17:29.225960 3971 ts_tablet_manager.cc:1939] T 4ec704647e92449a8e0520a298865f29 P a440dc1946004e438c3cc04e0776c55a: Deleting consensus metadata
I20260811 05:17:29.227279 3220 catalog_manager.cc:4841] TS a440dc1946004e438c3cc04e0776c55a (127.7.67.130:44825): tablet 4ec704647e92449a8e0520a298865f29 (table test-table [id=2be60d8b168144c58ba0989622222886]) successfully deleted
I20260811 05:17:29.227944 3974 ts_tablet_manager.cc:1918] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:17:29.228129 3974 log.cc:1196] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/4ec704647e92449a8e0520a298865f29
I20260811 05:17:29.228673 3974 ts_tablet_manager.cc:1939] T 4ec704647e92449a8e0520a298865f29 P b8c68b27658f4220bd0b15f51635197f: Deleting consensus metadata
W20260811 05:17:29.228806 3216 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.129:45117 (error 108)
I20260811 05:17:29.229913 3925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:29.229592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:45117 (local address 127.0.0.1:43806)
0811 05:17:29.229745 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:29.229820 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.129:45117: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 05:17:29.230016 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 3517
I20260811 05:17:29.229897 3217 catalog_manager.cc:4841] TS b8c68b27658f4220bd0b15f51635197f (127.7.67.131:38141): tablet 4ec704647e92449a8e0520a298865f29 (table test-table [id=2be60d8b168144c58ba0989622222886]) successfully deleted
W20260811 05:17:29.234534 3216 catalog_manager.cc:4584] TS 77f40239b3824ab296b54f19b2ff74c0 (127.7.67.129:45117): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4ec704647e92449a8e0520a298865f29: Network error: Client connection negotiation failed: client connection to 127.7.67.129:45117: connect: Connection refused (error 111)
I20260811 05:17:29.243505 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 3719
I20260811 05:17:29.255374 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 3195
2026-08-11T05:17:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1279 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:29 dist-test-slave-04bb krb5kdc[3978](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:29 dist-test-slave-04bb krb5kdc[3978](info): set up 2 sockets
Aug 11 05:17:29 dist-test-slave-04bb krb5kdc[3978](info): commencing operation
krb5kdc: starting...
W20260811 05:17:30.532473 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:17:30 dist-test-slave-04bb krb5kdc[3978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:17:30.683856 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:46279
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:46279
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:30.798202 4001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:30.798527 4001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:30.798609 4001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:30.802712 4001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:30.802806 4001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:30.802845 4001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:30.802904 4001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:30.802945 4001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:30.808260 4001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:46279
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:46279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:30.809468 4001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:30.810624 4001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:30.819494 4013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:30.819514 4011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:30.819511 4010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:30.819962 4001 server_base.cc:1034] running on GCE node
I20260811 05:17:30.820705 4001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:30.821950 4001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:30.823190 4001 hybrid_clock.cc:648] HybridClock initialized: now 1786425450823159 us; error 47 us; skew 500 ppm
Aug 11 05:17:30 dist-test-slave-04bb krb5kdc[3978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425450, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:30.826236 4001 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:17:30.827188 4001 webserver.cc:460] Webserver started at http://127.7.67.190:44927/ using document root <none> and password file <none>
I20260811 05:17:30.827795 4001 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:30.827903 4001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:30.828163 4001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:30.830431 4001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "cac52ad9a8154754a4478e2458e8a376"
format_stamp: "Formatted at 2026-08-11 05:17:30 on dist-test-slave-04bb"
server_key: "fa467f1e985ba98f1122da89eb195955"
server_key_iv: "87a83cf27d81649ed7b81811ed554340"
server_key_version: "encryptionkey@0"
I20260811 05:17:30.831007 4001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "cac52ad9a8154754a4478e2458e8a376"
format_stamp: "Formatted at 2026-08-11 05:17:30 on dist-test-slave-04bb"
server_key: "fa467f1e985ba98f1122da89eb195955"
server_key_iv: "87a83cf27d81649ed7b81811ed554340"
server_key_version: "encryptionkey@0"
I20260811 05:17:30.835131 4001 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 05:17:30.837807 4020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:30.839152 4001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 05:17:30.839321 4001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "cac52ad9a8154754a4478e2458e8a376"
format_stamp: "Formatted at 2026-08-11 05:17:30 on dist-test-slave-04bb"
server_key: "fa467f1e985ba98f1122da89eb195955"
server_key_iv: "87a83cf27d81649ed7b81811ed554340"
server_key_version: "encryptionkey@0"
I20260811 05:17:30.839443 4001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:30.852296 4001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:30.855566 4001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:30.855778 4001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:30.863230 4001 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:46279
I20260811 05:17:30.863263 4111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:46279 every 8 connection(s)
I20260811 05:17:30.864282 4001 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:30.867326 4113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:30.870594 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 4001
I20260811 05:17:30.870733 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:17:30.871073 7438 external_mini_cluster.cc:1428] Setting key d06c5534b27183a53b08f0a3c133737f
I20260811 05:17:30.873857 4113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376: Bootstrap starting.
I20260811 05:17:30.876750 4113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:30.877632 4113 log.cc:824] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:30.880024 4113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376: No bootstrap required, opened a new log
Aug 11 05:17:30 dist-test-slave-04bb krb5kdc[3978](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425450, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:17:30.884683 4113 raft_consensus.cc:348] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac52ad9a8154754a4478e2458e8a376" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 46279 } }
I20260811 05:17:30.885003 4113 raft_consensus.cc:374] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:30.885078 4113 raft_consensus.cc:729] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cac52ad9a8154754a4478e2458e8a376, State: Initialized, Role: FOLLOWER
I20260811 05:17:30.885646 4113 consensus_queue.cc:260] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac52ad9a8154754a4478e2458e8a376" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 46279 } }
I20260811 05:17:30.885836 4113 raft_consensus.cc:388] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:30.885910 4113 raft_consensus.cc:482] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:30.886013 4113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:30.887292 4113 raft_consensus.cc:504] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac52ad9a8154754a4478e2458e8a376" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 46279 } }
I20260811 05:17:30.887667 4113 leader_election.cc:302] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cac52ad9a8154754a4478e2458e8a376; no voters:
I20260811 05:17:30.888072 4113 leader_election.cc:288] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:30.888193 4119 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:30.888260 4116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:30.872745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56664 (local address 127.7.67.190:46279)
0811 05:17:30.873222 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:30.873243 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:30.873289 (+ 46us) server_negotiation.cc:407] Connection header received
0811 05:17:30.874121 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:30.874146 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:30.874571 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:30.874951 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:30.875675 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:30.877205 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:30.878552 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:30.879238 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:30.879426 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:30.882474 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:30.882484 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:30.882508 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:30.882540 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:30.884743 (+ 2203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:30.885302 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:30.885309 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:30.885314 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:30.885458 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:30.885755 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:30.885758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:30.885760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:30.886330 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:30.886524 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:30.886838 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 05:17:30.887200 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":124,"threads_started":1}
I20260811 05:17:30.888473 4119 raft_consensus.cc:686] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376 [term 1 LEADER]: Becoming Leader. State: Replica: cac52ad9a8154754a4478e2458e8a376, State: Running, Role: LEADER
I20260811 05:17:30.888810 4119 consensus_queue.cc:237] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac52ad9a8154754a4478e2458e8a376" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 46279 } }
I20260811 05:17:30.889473 4113 sys_catalog.cc:564] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:30.890009 4119 sys_catalog.cc:455] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cac52ad9a8154754a4478e2458e8a376" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac52ad9a8154754a4478e2458e8a376" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 46279 } } }
I20260811 05:17:30.890130 4119 sys_catalog.cc:458] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:30.890352 4119 sys_catalog.cc:455] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cac52ad9a8154754a4478e2458e8a376. Latest consensus state: current_term: 1 leader_uuid: "cac52ad9a8154754a4478e2458e8a376" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cac52ad9a8154754a4478e2458e8a376" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 46279 } } }
I20260811 05:17:30.890444 4119 sys_catalog.cc:458] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376 [sys.catalog]: This master's current role is: LEADER
W20260811 05:17:30.894582 4144 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:17:30.894672 4144 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:17:30.894758 4127 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:30.895454 4127 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:30.901192 4127 catalog_manager.cc:1347] Generated new cluster ID: e739bda393bf4faab22c8fdafd79074d
I20260811 05:17:30.901262 4127 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:30.930768 4127 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:30.931826 4127 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:30.943207 4127 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376: Generated new TSK 0
I20260811 05:17:30.944072 4127 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
I20260811 05:17:30.980094 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:46279
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:31.099748 4151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:31.100003 4151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:31.100075 4151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:31.104090 4151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:31.104146 4151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:31.104247 4151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:17:31.109653 4151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:46279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:31.110706 4151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:31.111749 4151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:31.121446 4156 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:31.121517 4161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:31.121456 4157 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:31.121732 4151 server_base.cc:1034] running on GCE node
I20260811 05:17:31.122170 4151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:31.122802 4151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:31.123996 4151 hybrid_clock.cc:648] HybridClock initialized: now 1786425451123978 us; error 31 us; skew 500 ppm
Aug 11 05:17:31 dist-test-slave-04bb krb5kdc[3978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425451, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:31.127177 4151 init.cc:377] Logged in from keytab as kudu/127.7.67.129@KRBTEST.COM (short username kudu)
I20260811 05:17:31.128230 4151 webserver.cc:460] Webserver started at http://127.7.67.129:37683/ using document root <none> and password file <none>
I20260811 05:17:31.128944 4151 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:31.129047 4151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:31.129318 4151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:31.131564 4151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "c0454a5b6cab492c8e94834748dfa32a"
format_stamp: "Formatted at 2026-08-11 05:17:31 on dist-test-slave-04bb"
server_key: "c7b13c2f526786d99f4b49db9d928628"
server_key_iv: "5e7c006ff78fe84ebb3490bee304d877"
server_key_version: "encryptionkey@0"
I20260811 05:17:31.132184 4151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "c0454a5b6cab492c8e94834748dfa32a"
format_stamp: "Formatted at 2026-08-11 05:17:31 on dist-test-slave-04bb"
server_key: "c7b13c2f526786d99f4b49db9d928628"
server_key_iv: "5e7c006ff78fe84ebb3490bee304d877"
server_key_version: "encryptionkey@0"
I20260811 05:17:31.136315 4151 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 05:17:31.139086 4170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:31.140482 4151 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 05:17:31.140684 4151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "c0454a5b6cab492c8e94834748dfa32a"
format_stamp: "Formatted at 2026-08-11 05:17:31 on dist-test-slave-04bb"
server_key: "c7b13c2f526786d99f4b49db9d928628"
server_key_iv: "5e7c006ff78fe84ebb3490bee304d877"
server_key_version: "encryptionkey@0"
I20260811 05:17:31.140810 4151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:31.153960 4151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:31.157262 4151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:31.157476 4151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:31.158102 4151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:31.159015 4151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:31.159093 4151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:31.159175 4151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:31.159222 4151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:31.171090 4151 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:37639
I20260811 05:17:31.171155 4344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:37639 every 8 connection(s)
I20260811 05:17:31.172106 4151 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:17:31.177444 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 4151
I20260811 05:17:31.177582 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:17:31.177934 7438 external_mini_cluster.cc:1428] Setting key ed9b1605784dacf3b56163f1b7b8ac02
Aug 11 05:17:31 dist-test-slave-04bb krb5kdc[3978](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425451, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:17:31.187951 4116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:31.174079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:56575 (local address 127.7.67.190:46279)
0811 05:17:31.174260 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:31.174264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:31.175077 (+ 813us) server_negotiation.cc:407] Connection header received
0811 05:17:31.176480 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:31.176483 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:31.176567 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:31.176671 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:31.178044 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.178735 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.180069 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.180468 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.180648 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:31.183818 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:31.183822 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:31.183824 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:31.183848 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:31.185673 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:31.186247 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:31.186252 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:31.186254 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:31.186352 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:31.186742 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:31.186746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:31.186747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:31.186950 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:31.187060 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:31.187582 (+ 522us) server_negotiation.cc:299] Negotiation successful
0811 05:17:31.187738 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260811 05:17:31.188776 4352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:31.174443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:46279 (local address 127.7.67.129:56575)
0811 05:17:31.174925 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:31.174968 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:31.176264 (+ 1296us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:31.176872 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:31.176878 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:31.177226 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:31.177802 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:31.177816 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.178889 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.178892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:31.179935 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:31.179945 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.180703 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.180710 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:31.180928 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:31.181651 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:31.181677 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:31.183617 (+ 1940us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:31.185806 (+ 2189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:31.185813 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:31.185827 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:31.186127 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:31.186462 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:31.186466 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:31.186468 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:31.186621 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:31.187078 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:31.187086 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:31.187332 (+ 246us) client_negotiation.cc:769] Sending connection context
0811 05:17:31.187533 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 05:17:31.187853 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":124,"threads_started":1}
WARNING: no policy specified for kudu/127.7.67.130@KRBTEST.COM; defaulting to no policy
I20260811 05:17:31.190181 4345 heartbeater.cc:347] Connected to a master server at 127.7.67.190:46279
I20260811 05:17:31.190446 4345 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:31.191124 4345 heartbeater.cc:511] Master 127.7.67.190:46279 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.130@KRBTEST.COM" created.
I20260811 05:17:31.192903 4049 ts_manager.cc:194] Registered new tserver with Master: c0454a5b6cab492c8e94834748dfa32a (127.7.67.129:37639)
I20260811 05:17:31.194046 4049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.129@KRBTEST.COM'} at 127.7.67.129:56575
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
I20260811 05:17:31.210050 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:46279
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:31.328193 4357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:31.328496 4357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:31.328589 4357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:31.332636 4357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:31.332716 4357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:31.332803 4357 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:17:31.338174 4357 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:46279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:31.339308 4357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:31.340361 4357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:31.350368 4365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:31.350430 4366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:31.350400 4369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:31.350725 4357 server_base.cc:1034] running on GCE node
I20260811 05:17:31.351109 4357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:31.351743 4357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:31.352905 4357 hybrid_clock.cc:648] HybridClock initialized: now 1786425451352867 us; error 45 us; skew 500 ppm
Aug 11 05:17:31 dist-test-slave-04bb krb5kdc[3978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425451, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:31.355580 4357 init.cc:377] Logged in from keytab as kudu/127.7.67.130@KRBTEST.COM (short username kudu)
I20260811 05:17:31.356465 4357 webserver.cc:460] Webserver started at http://127.7.67.130:40801/ using document root <none> and password file <none>
I20260811 05:17:31.357069 4357 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:31.357131 4357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:31.357306 4357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:31.359401 4357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "2b53556e539642228b8c81c6ca98ba7a"
format_stamp: "Formatted at 2026-08-11 05:17:31 on dist-test-slave-04bb"
server_key: "54ee5d8adcf0ab5ec7acadffced003b2"
server_key_iv: "5957efcb31dad41fc1981292fde66551"
server_key_version: "encryptionkey@0"
I20260811 05:17:31.359920 4357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "2b53556e539642228b8c81c6ca98ba7a"
format_stamp: "Formatted at 2026-08-11 05:17:31 on dist-test-slave-04bb"
server_key: "54ee5d8adcf0ab5ec7acadffced003b2"
server_key_iv: "5957efcb31dad41fc1981292fde66551"
server_key_version: "encryptionkey@0"
I20260811 05:17:31.363691 4357 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 05:17:31.366219 4379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:31.367476 4357 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:17:31.367585 4357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "2b53556e539642228b8c81c6ca98ba7a"
format_stamp: "Formatted at 2026-08-11 05:17:31 on dist-test-slave-04bb"
server_key: "54ee5d8adcf0ab5ec7acadffced003b2"
server_key_iv: "5957efcb31dad41fc1981292fde66551"
server_key_version: "encryptionkey@0"
I20260811 05:17:31.367683 4357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:31.384017 4357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:31.387410 4357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:31.387632 4357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:31.388299 4357 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:31.389308 4357 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:31.389379 4357 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:31.389454 4357 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:31.389508 4357 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:31.399392 4357 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:34677
I20260811 05:17:31.399461 4567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:34677 every 8 connection(s)
I20260811 05:17:31.400384 4357 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:17:31.407968 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 4357
I20260811 05:17:31.408109 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:17:31.408429 7438 external_mini_cluster.cc:1428] Setting key 7ec477a0f6da8174ed8687d5e4fa2998
Aug 11 05:17:31 dist-test-slave-04bb krb5kdc[3978](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425451, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:17:31.416867 4116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:31.402342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:48607 (local address 127.7.67.190:46279)
0811 05:17:31.402492 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:31.402496 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:31.403395 (+ 899us) server_negotiation.cc:407] Connection header received
0811 05:17:31.404921 (+ 1526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:31.404925 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:31.404988 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:31.405106 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:31.406366 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.406932 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.408984 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.409336 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.409466 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:31.412614 (+ 3148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:31.412620 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:31.412625 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:31.412672 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:31.414420 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:31.414943 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:31.414949 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:31.414954 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:31.415054 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:31.415435 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:31.415441 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:31.415446 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:31.415747 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:31.415866 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:31.416421 (+ 555us) server_negotiation.cc:299] Negotiation successful
0811 05:17:31.416606 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 05:17:31.417683 4571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:31.402744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:46279 (local address 127.7.67.130:48607)
0811 05:17:31.403226 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:31.403273 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:31.404693 (+ 1420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:31.405293 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:31.405300 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:31.405682 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:31.406202 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:31.406216 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.407078 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.407090 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:31.408826 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:31.408838 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.409506 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.409510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:31.409702 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:31.410465 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:31.410482 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:31.412401 (+ 1919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:31.414555 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:31.414560 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:31.414572 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:31.414836 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:31.415155 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:31.415157 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:31.415159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:31.415323 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:31.415865 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:31.415870 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:31.416115 (+ 245us) client_negotiation.cc:769] Sending connection context
0811 05:17:31.416349 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 05:17:31.416730 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":116,"threads_started":1}
I20260811 05:17:31.418953 4568 heartbeater.cc:347] Connected to a master server at 127.7.67.190:46279
I20260811 05:17:31.419206 4568 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:31.419736 4568 heartbeater.cc:511] Master 127.7.67.190:46279 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.67.131@KRBTEST.COM; defaulting to no policy
I20260811 05:17:31.420917 4049 ts_manager.cc:194] Registered new tserver with Master: 2b53556e539642228b8c81c6ca98ba7a (127.7.67.130:34677)
I20260811 05:17:31.421519 4049 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.130@KRBTEST.COM'} at 127.7.67.130:48607
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
I20260811 05:17:31.439687 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:46279
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:31.557224 4574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:31.557474 4574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:31.557516 4574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:31.561476 4574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:31.561527 4574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:31.561581 4574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:17:31.566594 4574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:46279
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:31.567549 4574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:31.568560 4574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:31.578245 4583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:31.578250 4582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:31.578378 4585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:31.578648 4574 server_base.cc:1034] running on GCE node
I20260811 05:17:31.579087 4574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:31.579654 4574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:31.580812 4574 hybrid_clock.cc:648] HybridClock initialized: now 1786425451580794 us; error 34 us; skew 500 ppm
Aug 11 05:17:31 dist-test-slave-04bb krb5kdc[3978](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425451, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:31.583606 4574 init.cc:377] Logged in from keytab as kudu/127.7.67.131@KRBTEST.COM (short username kudu)
I20260811 05:17:31.584568 4574 webserver.cc:460] Webserver started at http://127.7.67.131:41895/ using document root <none> and password file <none>
I20260811 05:17:31.585146 4574 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:31.585239 4574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:31.585516 4574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:31.587606 4574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "e8b81aebf3a8467594dcdedaf1ef776c"
format_stamp: "Formatted at 2026-08-11 05:17:31 on dist-test-slave-04bb"
server_key: "af073af6cba772d81dc079cf2390b2b3"
server_key_iv: "ef3b84f64d0362bfa72d83d9757fb530"
server_key_version: "encryptionkey@0"
I20260811 05:17:31.588186 4574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "e8b81aebf3a8467594dcdedaf1ef776c"
format_stamp: "Formatted at 2026-08-11 05:17:31 on dist-test-slave-04bb"
server_key: "af073af6cba772d81dc079cf2390b2b3"
server_key_iv: "ef3b84f64d0362bfa72d83d9757fb530"
server_key_version: "encryptionkey@0"
I20260811 05:17:31.592133 4574 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 05:17:31.594621 4594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:31.595849 4574 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 05:17:31.595993 4574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "e8b81aebf3a8467594dcdedaf1ef776c"
format_stamp: "Formatted at 2026-08-11 05:17:31 on dist-test-slave-04bb"
server_key: "af073af6cba772d81dc079cf2390b2b3"
server_key_iv: "ef3b84f64d0362bfa72d83d9757fb530"
server_key_version: "encryptionkey@0"
I20260811 05:17:31.596102 4574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:31.609436 4574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:31.612393 4574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:31.612648 4574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:31.613299 4574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:31.614118 4574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:31.614192 4574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:31.614269 4574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:31.614312 4574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:31.623977 4574 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:43451
I20260811 05:17:31.624017 4758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:43451 every 8 connection(s)
I20260811 05:17:31.624935 4574 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:17:31.627198 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 4574
I20260811 05:17:31.627339 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:17:31.627694 7438 external_mini_cluster.cc:1428] Setting key 852d10dce18d58f237ea53e509ba9899
Aug 11 05:17:31 dist-test-slave-04bb krb5kdc[3978](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425451, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:17:31.639247 4116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:31.626840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:32807 (local address 127.7.67.190:46279)
0811 05:17:31.626998 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:31.627003 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:31.628044 (+ 1041us) server_negotiation.cc:407] Connection header received
0811 05:17:31.629733 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:31.629737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:31.629816 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:31.629955 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:31.631105 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.631624 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.632826 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.633037 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.633119 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:31.635502 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:31.635505 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:31.635506 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:31.635530 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:31.637146 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:31.637846 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:31.637849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:31.637850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:31.637924 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:31.638253 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:31.638255 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:31.638256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:31.638438 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:31.638554 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:31.639021 (+ 467us) server_negotiation.cc:299] Negotiation successful
0811 05:17:31.639122 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:17:31.640216 4761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:31.627292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:46279 (local address 127.7.67.131:32807)
0811 05:17:31.627854 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:31.627925 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:31.629521 (+ 1596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:31.630134 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:31.630141 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:31.630495 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:31.630976 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:31.630989 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.631743 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.631746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:31.632716 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:31.632725 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.633169 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.633172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:31.633309 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:31.633871 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:31.633886 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:31.635336 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:31.637422 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:31.637427 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:31.637439 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:31.637740 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:31.638009 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:31.638011 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:31.638013 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:31.638166 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:31.638556 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:31.638561 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:31.638800 (+ 239us) client_negotiation.cc:769] Sending connection context
0811 05:17:31.638969 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 05:17:31.639272 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":118,"threads_started":1}
I20260811 05:17:31.641515 4759 heartbeater.cc:347] Connected to a master server at 127.7.67.190:46279
I20260811 05:17:31.641789 4759 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:31.642343 4759 heartbeater.cc:511] Master 127.7.67.190:46279 requested a full tablet report, sending...
I20260811 05:17:31.643489 4048 ts_manager.cc:194] Registered new tserver with Master: e8b81aebf3a8467594dcdedaf1ef776c (127.7.67.131:43451)
I20260811 05:17:31.644057 4048 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.131@KRBTEST.COM'} at 127.7.67.131:32807
I20260811 05:17:31.653843 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:17:31.664040 4116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:31.656372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56676 (local address 127.7.67.190:46279)
0811 05:17:31.656498 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:31.656503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:31.656724 (+ 221us) server_negotiation.cc:407] Connection header received
0811 05:17:31.656808 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:31.656810 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:31.656850 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:31.656920 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:31.657613 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.658134 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.658967 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.659176 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.659266 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:31.660922 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:31.660926 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:31.660928 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:31.660951 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:31.662460 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:31.662938 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:31.662940 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:31.662941 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:31.663012 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:31.663300 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:31.663302 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:31.663303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:31.663497 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:31.663609 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:31.663843 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 05:17:31.663921 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 05:17:31.667366 4048 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56676:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:17:31.670475 4048 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:17:31.685672 4786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:31.680321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38766 (local address 127.7.67.131:43451)
0811 05:17:31.680981 (+ 660us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:31.680994 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:31.681016 (+ 22us) server_negotiation.cc:407] Connection header received
0811 05:17:31.681092 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:31.681098 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:31.681346 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:31.681524 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:31.681771 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.682922 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.684303 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.684981 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.685109 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:31.685284 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:31.685424 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 05:17:31.685512 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":552,"thread_start_us":143,"threads_started":1}
I20260811 05:17:31.685667 4778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:31.679673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:43451 (local address 127.0.0.1:38766)
0811 05:17:31.680374 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:31.680426 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:31.680582 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:31.681511 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:31.681514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:31.681532 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:31.681628 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:31.681637 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.683069 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.683072 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:31.684177 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:31.684185 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.685112 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.685116 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:31.685234 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:31.685275 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 05:17:31.685394 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 05:17:31.685513 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"spinlock_wait_cycles":15616,"thread_start_us":104,"threads_started":1}
I20260811 05:17:31.686013 4782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:31.679665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46368 (local address 127.7.67.129:37639)
0811 05:17:31.680080 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:31.680097 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:31.680546 (+ 449us) server_negotiation.cc:407] Connection header received
0811 05:17:31.680817 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:31.680829 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:31.681055 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:31.681215 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:31.681734 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.682559 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.684151 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.685004 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.685113 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:31.685277 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:31.685738 (+ 461us) server_negotiation.cc:299] Negotiation successful
0811 05:17:31.685854 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":109,"threads_started":1}
I20260811 05:17:31.686271 4781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:31.679458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:37639 (local address 127.0.0.1:46368)
0811 05:17:31.680375 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:31.680429 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:31.680628 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:31.681399 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:31.681405 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:31.681428 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:31.681582 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:31.681590 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.682689 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.682694 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:31.684008 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:31.684019 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.685350 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.685357 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:31.685445 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:31.685468 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 05:17:31.685541 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 05:17:31.685612 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":739,"thread_start_us":76,"threads_started":1}
I20260811 05:17:31.686473 4784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:31.679878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:34677 (local address 127.0.0.1:54704)
0811 05:17:31.680790 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:31.680804 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:31.680879 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:31.681745 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:31.681748 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:31.681766 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:31.681859 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:31.681864 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.683220 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.683223 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:31.684325 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:31.684335 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.685575 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.685578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:31.685671 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:31.685697 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 05:17:31.686243 (+ 546us) client_negotiation.cc:240] Negotiation successful
0811 05:17:31.686327 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"thread_start_us":87,"threads_started":1}
I20260811 05:17:31.686882 4785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:31.680288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54704 (local address 127.7.67.130:34677)
0811 05:17:31.681129 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:31.681142 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:31.681164 (+ 22us) server_negotiation.cc:407] Connection header received
0811 05:17:31.681247 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:31.681254 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:31.681428 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:31.681574 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:31.681987 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.683089 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.684456 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:31.685286 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:31.685390 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:31.685517 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:31.686634 (+ 1117us) server_negotiation.cc:299] Negotiation successful
0811 05:17:31.686738 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":728,"thread_start_us":170,"threads_started":1}
I20260811 05:17:31.689096 4248 tablet_service.cc:1467] Processing CreateTablet for tablet d37125a573c341458e31a7097894ac47 (DEFAULT_TABLE table=test-table [id=4eece7aeccb743d3be32a037e3f03e90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:31.689131 4669 tablet_service.cc:1467] Processing CreateTablet for tablet d37125a573c341458e31a7097894ac47 (DEFAULT_TABLE table=test-table [id=4eece7aeccb743d3be32a037e3f03e90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:31.690225 4248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d37125a573c341458e31a7097894ac47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:31.690261 4669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d37125a573c341458e31a7097894ac47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:31.690093 4461 tablet_service.cc:1467] Processing CreateTablet for tablet d37125a573c341458e31a7097894ac47 (DEFAULT_TABLE table=test-table [id=4eece7aeccb743d3be32a037e3f03e90]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:31.691035 4461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d37125a573c341458e31a7097894ac47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:31.696110 4789 tablet_bootstrap.cc:492] T d37125a573c341458e31a7097894ac47 P c0454a5b6cab492c8e94834748dfa32a: Bootstrap starting.
I20260811 05:17:31.698375 4792 tablet_bootstrap.cc:492] T d37125a573c341458e31a7097894ac47 P 2b53556e539642228b8c81c6ca98ba7a: Bootstrap starting.
I20260811 05:17:31.698444 4789 tablet_bootstrap.cc:654] T d37125a573c341458e31a7097894ac47 P c0454a5b6cab492c8e94834748dfa32a: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:31.698954 4791 tablet_bootstrap.cc:492] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c: Bootstrap starting.
I20260811 05:17:31.699419 4789 log.cc:824] T d37125a573c341458e31a7097894ac47 P c0454a5b6cab492c8e94834748dfa32a: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:31.700549 4792 tablet_bootstrap.cc:654] T d37125a573c341458e31a7097894ac47 P 2b53556e539642228b8c81c6ca98ba7a: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:31.701149 4791 tablet_bootstrap.cc:654] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:31.701597 4789 tablet_bootstrap.cc:492] T d37125a573c341458e31a7097894ac47 P c0454a5b6cab492c8e94834748dfa32a: No bootstrap required, opened a new log
I20260811 05:17:31.701803 4789 ts_tablet_manager.cc:1397] T d37125a573c341458e31a7097894ac47 P c0454a5b6cab492c8e94834748dfa32a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 05:17:31.701938 4792 log.cc:824] T d37125a573c341458e31a7097894ac47 P 2b53556e539642228b8c81c6ca98ba7a: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:31.702847 4791 log.cc:824] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:31.703981 4792 tablet_bootstrap.cc:492] T d37125a573c341458e31a7097894ac47 P 2b53556e539642228b8c81c6ca98ba7a: No bootstrap required, opened a new log
I20260811 05:17:31.704164 4792 ts_tablet_manager.cc:1397] T d37125a573c341458e31a7097894ac47 P 2b53556e539642228b8c81c6ca98ba7a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 05:17:31.704924 4791 tablet_bootstrap.cc:492] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c: No bootstrap required, opened a new log
I20260811 05:17:31.705111 4791 ts_tablet_manager.cc:1397] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:17:31.705922 4789 raft_consensus.cc:348] T d37125a573c341458e31a7097894ac47 P c0454a5b6cab492c8e94834748dfa32a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0454a5b6cab492c8e94834748dfa32a" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 37639 } } peers { permanent_uuid: "e8b81aebf3a8467594dcdedaf1ef776c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43451 } } peers { permanent_uuid: "2b53556e539642228b8c81c6ca98ba7a" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 34677 } }
I20260811 05:17:31.706161 4789 raft_consensus.cc:374] T d37125a573c341458e31a7097894ac47 P c0454a5b6cab492c8e94834748dfa32a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:31.706245 4789 raft_consensus.cc:729] T d37125a573c341458e31a7097894ac47 P c0454a5b6cab492c8e94834748dfa32a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0454a5b6cab492c8e94834748dfa32a, State: Initialized, Role: FOLLOWER
I20260811 05:17:31.706714 4789 consensus_queue.cc:260] T d37125a573c341458e31a7097894ac47 P c0454a5b6cab492c8e94834748dfa32a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0454a5b6cab492c8e94834748dfa32a" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 37639 } } peers { permanent_uuid: "e8b81aebf3a8467594dcdedaf1ef776c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43451 } } peers { permanent_uuid: "2b53556e539642228b8c81c6ca98ba7a" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 34677 } }
I20260811 05:17:31.707796 4789 ts_tablet_manager.cc:1428] T d37125a573c341458e31a7097894ac47 P c0454a5b6cab492c8e94834748dfa32a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 05:17:31.707829 4345 heartbeater.cc:503] Master 127.7.67.190:46279 was elected leader, sending a full tablet report...
I20260811 05:17:31.707975 4792 raft_consensus.cc:348] T d37125a573c341458e31a7097894ac47 P 2b53556e539642228b8c81c6ca98ba7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0454a5b6cab492c8e94834748dfa32a" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 37639 } } peers { permanent_uuid: "e8b81aebf3a8467594dcdedaf1ef776c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43451 } } peers { permanent_uuid: "2b53556e539642228b8c81c6ca98ba7a" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 34677 } }
I20260811 05:17:31.708259 4792 raft_consensus.cc:374] T d37125a573c341458e31a7097894ac47 P 2b53556e539642228b8c81c6ca98ba7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:31.708323 4792 raft_consensus.cc:729] T d37125a573c341458e31a7097894ac47 P 2b53556e539642228b8c81c6ca98ba7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b53556e539642228b8c81c6ca98ba7a, State: Initialized, Role: FOLLOWER
I20260811 05:17:31.708778 4792 consensus_queue.cc:260] T d37125a573c341458e31a7097894ac47 P 2b53556e539642228b8c81c6ca98ba7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0454a5b6cab492c8e94834748dfa32a" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 37639 } } peers { permanent_uuid: "e8b81aebf3a8467594dcdedaf1ef776c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43451 } } peers { permanent_uuid: "2b53556e539642228b8c81c6ca98ba7a" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 34677 } }
I20260811 05:17:31.709491 4791 raft_consensus.cc:348] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0454a5b6cab492c8e94834748dfa32a" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 37639 } } peers { permanent_uuid: "e8b81aebf3a8467594dcdedaf1ef776c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43451 } } peers { permanent_uuid: "2b53556e539642228b8c81c6ca98ba7a" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 34677 } }
I20260811 05:17:31.709738 4791 raft_consensus.cc:374] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:31.709790 4791 raft_consensus.cc:729] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8b81aebf3a8467594dcdedaf1ef776c, State: Initialized, Role: FOLLOWER
I20260811 05:17:31.709800 4792 ts_tablet_manager.cc:1428] T d37125a573c341458e31a7097894ac47 P 2b53556e539642228b8c81c6ca98ba7a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 05:17:31.710162 4791 consensus_queue.cc:260] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0454a5b6cab492c8e94834748dfa32a" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 37639 } } peers { permanent_uuid: "e8b81aebf3a8467594dcdedaf1ef776c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43451 } } peers { permanent_uuid: "2b53556e539642228b8c81c6ca98ba7a" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 34677 } }
I20260811 05:17:31.710803 4759 heartbeater.cc:503] Master 127.7.67.190:46279 was elected leader, sending a full tablet report...
I20260811 05:17:31.710870 4568 heartbeater.cc:503] Master 127.7.67.190:46279 was elected leader, sending a full tablet report...
I20260811 05:17:31.710976 4791 ts_tablet_manager.cc:1428] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260811 05:17:31.876075 4760 tablet.cc:2369] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:17:31.901645 4570 tablet.cc:2369] T d37125a573c341458e31a7097894ac47 P 2b53556e539642228b8c81c6ca98ba7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 05:17:31.923815 4351 tablet.cc:2369] T d37125a573c341458e31a7097894ac47 P c0454a5b6cab492c8e94834748dfa32a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:17:32.155381 4799 raft_consensus.cc:482] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:17:32.155659 4799 raft_consensus.cc:504] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0454a5b6cab492c8e94834748dfa32a" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 37639 } } peers { permanent_uuid: "e8b81aebf3a8467594dcdedaf1ef776c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43451 } } peers { permanent_uuid: "2b53556e539642228b8c81c6ca98ba7a" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 34677 } }
I20260811 05:17:32.157040 4799 leader_election.cc:288] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c0454a5b6cab492c8e94834748dfa32a (127.7.67.129:37639), 2b53556e539642228b8c81c6ca98ba7a (127.7.67.130:34677)
I20260811 05:17:32.162955 4785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:32.157522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:59633 (local address 127.7.67.130:34677)
0811 05:17:32.157682 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:32.157686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:32.157903 (+ 217us) server_negotiation.cc:407] Connection header received
0811 05:17:32.158087 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:32.158090 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:32.158146 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:32.158236 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:32.158797 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:32.159578 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:32.161569 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:32.162190 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:32.162327 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:32.162381 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:32.162752 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 05:17:32.162823 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 05:17:32.162955 4782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:32.157428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:38961 (local address 127.7.67.129:37639)
0811 05:17:32.157586 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:32.157591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:32.157994 (+ 403us) server_negotiation.cc:407] Connection header received
0811 05:17:32.158092 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:32.158094 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:32.158146 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:32.158269 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:32.158797 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:32.159598 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:32.161569 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:32.162190 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:32.162341 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:32.162382 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:32.162752 (+ 370us) server_negotiation.cc:299] Negotiation successful
0811 05:17:32.162823 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 05:17:32.163033 4801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:32.157311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:34677 (local address 127.7.67.131:59633)
0811 05:17:32.157785 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:32.157803 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:32.157941 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:32.158407 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:32.158411 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:32.158459 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:32.158651 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:32.158658 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:32.159719 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:32.159722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:32.161418 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:32.161431 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:32.162356 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:32.162361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:32.162539 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:32.162622 (+ 83us) client_negotiation.cc:769] Sending connection context
0811 05:17:32.162713 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 05:17:32.162837 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":255,"spinlock_wait_cycles":47104,"thread_start_us":130,"threads_started":1}
I20260811 05:17:32.163038 4802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:32.157315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:37639 (local address 127.7.67.131:38961)
0811 05:17:32.157892 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:32.157906 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:32.158010 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:32.158407 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:32.158411 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:32.158457 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:32.158651 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:32.158658 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:32.159720 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:32.159722 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:32.161418 (+ 1696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:32.161431 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:32.162356 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:32.162361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:32.162570 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:32.162631 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 05:17:32.162728 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 05:17:32.162838 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":338,"spinlock_wait_cycles":31872,"thread_start_us":74,"threads_started":1}
I20260811 05:17:32.163798 4492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d37125a573c341458e31a7097894ac47" candidate_uuid: "e8b81aebf3a8467594dcdedaf1ef776c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b53556e539642228b8c81c6ca98ba7a" is_pre_election: true
I20260811 05:17:32.163877 4279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d37125a573c341458e31a7097894ac47" candidate_uuid: "e8b81aebf3a8467594dcdedaf1ef776c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0454a5b6cab492c8e94834748dfa32a" is_pre_election: true
I20260811 05:17:32.164136 4492 raft_consensus.cc:2454] T d37125a573c341458e31a7097894ac47 P 2b53556e539642228b8c81c6ca98ba7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8b81aebf3a8467594dcdedaf1ef776c in term 0.
I20260811 05:17:32.164242 4279 raft_consensus.cc:2454] T d37125a573c341458e31a7097894ac47 P c0454a5b6cab492c8e94834748dfa32a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e8b81aebf3a8467594dcdedaf1ef776c in term 0.
I20260811 05:17:32.164742 4598 leader_election.cc:302] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b53556e539642228b8c81c6ca98ba7a, e8b81aebf3a8467594dcdedaf1ef776c; no voters:
I20260811 05:17:32.165047 4799 raft_consensus.cc:2781] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:17:32.165143 4799 raft_consensus.cc:482] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:17:32.165210 4799 raft_consensus.cc:3037] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:32.166364 4799 raft_consensus.cc:504] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0454a5b6cab492c8e94834748dfa32a" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 37639 } } peers { permanent_uuid: "e8b81aebf3a8467594dcdedaf1ef776c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43451 } } peers { permanent_uuid: "2b53556e539642228b8c81c6ca98ba7a" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 34677 } }
I20260811 05:17:32.166813 4799 leader_election.cc:288] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c [CANDIDATE]: Term 1 election: Requested vote from peers c0454a5b6cab492c8e94834748dfa32a (127.7.67.129:37639), 2b53556e539642228b8c81c6ca98ba7a (127.7.67.130:34677)
I20260811 05:17:32.167193 4279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d37125a573c341458e31a7097894ac47" candidate_uuid: "e8b81aebf3a8467594dcdedaf1ef776c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0454a5b6cab492c8e94834748dfa32a"
I20260811 05:17:32.167273 4492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d37125a573c341458e31a7097894ac47" candidate_uuid: "e8b81aebf3a8467594dcdedaf1ef776c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b53556e539642228b8c81c6ca98ba7a"
I20260811 05:17:32.167335 4279 raft_consensus.cc:3037] T d37125a573c341458e31a7097894ac47 P c0454a5b6cab492c8e94834748dfa32a [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:32.167423 4492 raft_consensus.cc:3037] T d37125a573c341458e31a7097894ac47 P 2b53556e539642228b8c81c6ca98ba7a [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:32.168519 4492 raft_consensus.cc:2454] T d37125a573c341458e31a7097894ac47 P 2b53556e539642228b8c81c6ca98ba7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8b81aebf3a8467594dcdedaf1ef776c in term 1.
I20260811 05:17:32.168776 4279 raft_consensus.cc:2454] T d37125a573c341458e31a7097894ac47 P c0454a5b6cab492c8e94834748dfa32a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e8b81aebf3a8467594dcdedaf1ef776c in term 1.
I20260811 05:17:32.169080 4598 leader_election.cc:302] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b53556e539642228b8c81c6ca98ba7a, e8b81aebf3a8467594dcdedaf1ef776c; no voters:
I20260811 05:17:32.169287 4799 raft_consensus.cc:2781] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:32.169512 4799 raft_consensus.cc:686] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c [term 1 LEADER]: Becoming Leader. State: Replica: e8b81aebf3a8467594dcdedaf1ef776c, State: Running, Role: LEADER
I20260811 05:17:32.169874 4799 consensus_queue.cc:237] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0454a5b6cab492c8e94834748dfa32a" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 37639 } } peers { permanent_uuid: "e8b81aebf3a8467594dcdedaf1ef776c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43451 } } peers { permanent_uuid: "2b53556e539642228b8c81c6ca98ba7a" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 34677 } }
I20260811 05:17:32.173249 4046 catalog_manager.cc:5495] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c reported cstate change: term changed from 0 to 1, leader changed from <none> to e8b81aebf3a8467594dcdedaf1ef776c (127.7.67.131). New cstate: current_term: 1 leader_uuid: "e8b81aebf3a8467594dcdedaf1ef776c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c0454a5b6cab492c8e94834748dfa32a" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 37639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e8b81aebf3a8467594dcdedaf1ef776c" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 43451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b53556e539642228b8c81c6ca98ba7a" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 34677 } health_report { overall_health: UNKNOWN } } }
I20260811 05:17:32.256376 4811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:32.252019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38770 (local address 127.7.67.131:43451)
0811 05:17:32.252243 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:32.252247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:32.252263 (+ 16us) server_negotiation.cc:407] Connection header received
0811 05:17:32.252410 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:32.252413 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:32.252473 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:32.252591 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:17:32.252815 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:32.253327 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:32.254251 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:32.254491 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:32.254570 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:32.255203 (+ 633us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:17:32.255793 (+ 590us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:17:32.255966 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:32.256171 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 05:17:32.256213 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":76,"threads_started":1}
I20260811 05:17:32.262825 4492 raft_consensus.cc:1261] T d37125a573c341458e31a7097894ac47 P 2b53556e539642228b8c81c6ca98ba7a [term 1 FOLLOWER]: Refusing update from remote peer e8b81aebf3a8467594dcdedaf1ef776c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:32.262821 4279 raft_consensus.cc:1261] T d37125a573c341458e31a7097894ac47 P c0454a5b6cab492c8e94834748dfa32a [term 1 FOLLOWER]: Refusing update from remote peer e8b81aebf3a8467594dcdedaf1ef776c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:32.263536 4799 consensus_queue.cc:1035] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b53556e539642228b8c81c6ca98ba7a" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 34677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:32.263795 4803 consensus_queue.cc:1035] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0454a5b6cab492c8e94834748dfa32a" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 37639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:32.271886 4815 mvcc.cc:205] Tried to move back new op lower bound from 7317198652460781568 to 7317198652090523648. Current Snapshot: MvccSnapshot[applied={T|T < 7317198652460781568}]
I20260811 05:17:32.273823 4817 mvcc.cc:205] Tried to move back new op lower bound from 7317198652460781568 to 7317198652090523648. Current Snapshot: MvccSnapshot[applied={T|T < 7317198652460781568}]
I20260811 05:17:32.282423 4049 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:17:32.282610 4049 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56676:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:17:32.285351 4049 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cac52ad9a8154754a4478e2458e8a376: Sending DeleteTablet for 3 replicas of tablet d37125a573c341458e31a7097894ac47
I20260811 05:17:32.286217 4669 tablet_service.cc:1514] Processing DeleteTablet for tablet d37125a573c341458e31a7097894ac47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:32 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:38766
I20260811 05:17:32.286216 4248 tablet_service.cc:1514] Processing DeleteTablet for tablet d37125a573c341458e31a7097894ac47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:32 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:46368
I20260811 05:17:32.286782 4835 tablet_replica.cc:330] T d37125a573c341458e31a7097894ac47 P c0454a5b6cab492c8e94834748dfa32a: stopping tablet replica
I20260811 05:17:32.286795 4461 tablet_service.cc:1514] Processing DeleteTablet for tablet d37125a573c341458e31a7097894ac47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:32 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:54704
I20260811 05:17:32.287282 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 4151
I20260811 05:17:32.287295 4836 tablet_replica.cc:330] T d37125a573c341458e31a7097894ac47 P 2b53556e539642228b8c81c6ca98ba7a: stopping tablet replica
I20260811 05:17:32.290467 4836 raft_consensus.cc:2229] T d37125a573c341458e31a7097894ac47 P 2b53556e539642228b8c81c6ca98ba7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:17:32.290926 4836 raft_consensus.cc:2258] T d37125a573c341458e31a7097894ac47 P 2b53556e539642228b8c81c6ca98ba7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:32.293949 4836 ts_tablet_manager.cc:1905] T d37125a573c341458e31a7097894ac47 P 2b53556e539642228b8c81c6ca98ba7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:17:32.297148 4836 ts_tablet_manager.cc:1918] T d37125a573c341458e31a7097894ac47 P 2b53556e539642228b8c81c6ca98ba7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:17:32.297259 4836 log.cc:1196] T d37125a573c341458e31a7097894ac47 P 2b53556e539642228b8c81c6ca98ba7a: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/d37125a573c341458e31a7097894ac47
I20260811 05:17:32.297607 4836 ts_tablet_manager.cc:1939] T d37125a573c341458e31a7097894ac47 P 2b53556e539642228b8c81c6ca98ba7a: Deleting consensus metadata
I20260811 05:17:32.299052 4026 catalog_manager.cc:4841] TS 2b53556e539642228b8c81c6ca98ba7a (127.7.67.130:34677): tablet d37125a573c341458e31a7097894ac47 (table test-table [id=4eece7aeccb743d3be32a037e3f03e90]) successfully deleted
I20260811 05:17:32.299708 4832 tablet_replica.cc:330] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c: stopping tablet replica
I20260811 05:17:32.300112 4832 raft_consensus.cc:2229] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:17:32.300652 4832 raft_consensus.cc:2258] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:32.302049 4832 ts_tablet_manager.cc:1905] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:17:32.304294 4832 ts_tablet_manager.cc:1918] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:17:32.304418 4832 log.cc:1196] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/d37125a573c341458e31a7097894ac47
I20260811 05:17:32.304749 4832 ts_tablet_manager.cc:1939] T d37125a573c341458e31a7097894ac47 P e8b81aebf3a8467594dcdedaf1ef776c: Deleting consensus metadata
I20260811 05:17:32.305883 4021 catalog_manager.cc:4841] TS e8b81aebf3a8467594dcdedaf1ef776c (127.7.67.131:43451): tablet d37125a573c341458e31a7097894ac47 (table test-table [id=4eece7aeccb743d3be32a037e3f03e90]) successfully deleted
W20260811 05:17:32.311177 4025 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.129:37639 (error 108)
I20260811 05:17:32.311362 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 4357
I20260811 05:17:32.327895 4841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:32.327424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:37639 (local address 127.0.0.1:46382)
0811 05:17:32.327699 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:32.327778 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.129:37639: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":101,"threads_started":1}
W20260811 05:17:32.328123 4025 catalog_manager.cc:4584] TS c0454a5b6cab492c8e94834748dfa32a (127.7.67.129:37639): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d37125a573c341458e31a7097894ac47: Network error: Client connection negotiation failed: client connection to 127.7.67.129:37639: connect: Connection refused (error 111)
I20260811 05:17:32.335225 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 4574
I20260811 05:17:32.359897 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 4001
2026-08-11T05:17:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (3108 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T05:17:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:32Z Disabled control of system clock
I20260811 05:17:32.399557 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:44655
--webserver_interface=127.7.67.190
--webserver_port=0
--builtin_ntp_servers=127.7.67.148:44843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:44655
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 05:17:32.514837 4850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:32.515147 4850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:32.515184 4850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:32.519546 4850 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:32.519623 4850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:32.519640 4850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:32.519657 4850 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:32.519673 4850 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:32.525496 4850 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:44843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:44655
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:44655
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:32.526678 4850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:32.528185 4850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:32.537020 4857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:32.537036 4856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:32.537103 4859 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:32.537484 4850 server_base.cc:1034] running on GCE node
I20260811 05:17:32.537973 4850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:32.539053 4850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:32.540216 4850 hybrid_clock.cc:648] HybridClock initialized: now 1786425452540196 us; error 31 us; skew 500 ppm
I20260811 05:17:32.548111 4850 webserver.cc:460] Webserver started at http://127.7.67.190:39719/ using document root <none> and password file <none>
I20260811 05:17:32.548789 4850 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:32.548861 4850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:32.549042 4850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:32.551105 4850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "a29dd7681fcc4ef0b7cc21e7a4d77b8e"
format_stamp: "Formatted at 2026-08-11 05:17:32 on dist-test-slave-04bb"
server_key: "48d869f67c9381103fb2b14fbbec8e6e"
server_key_iv: "12bd14561c9d00c2073eb6ae87927282"
server_key_version: "encryptionkey@0"
I20260811 05:17:32.551630 4850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "a29dd7681fcc4ef0b7cc21e7a4d77b8e"
format_stamp: "Formatted at 2026-08-11 05:17:32 on dist-test-slave-04bb"
server_key: "48d869f67c9381103fb2b14fbbec8e6e"
server_key_iv: "12bd14561c9d00c2073eb6ae87927282"
server_key_version: "encryptionkey@0"
I20260811 05:17:32.555501 4850 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 05:17:32.558324 4865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:32.559631 4850 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:17:32.559752 4850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "a29dd7681fcc4ef0b7cc21e7a4d77b8e"
format_stamp: "Formatted at 2026-08-11 05:17:32 on dist-test-slave-04bb"
server_key: "48d869f67c9381103fb2b14fbbec8e6e"
server_key_iv: "12bd14561c9d00c2073eb6ae87927282"
server_key_version: "encryptionkey@0"
I20260811 05:17:32.559842 4850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:32.577591 4850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:32.578275 4850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:32.578431 4850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:32.585758 4850 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:44655
I20260811 05:17:32.585790 4951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:44655 every 8 connection(s)
I20260811 05:17:32.586746 4850 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:32.590090 4952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:32.595773 4952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a29dd7681fcc4ef0b7cc21e7a4d77b8e: Bootstrap starting.
I20260811 05:17:32.596328 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 4850
I20260811 05:17:32.596457 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:17:32.596792 7438 external_mini_cluster.cc:1428] Setting key 62f243dc56b9ab3a15989b6591c6a444
I20260811 05:17:32.598754 4952 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a29dd7681fcc4ef0b7cc21e7a4d77b8e: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:32.599524 4952 log.cc:824] T 00000000000000000000000000000000 P a29dd7681fcc4ef0b7cc21e7a4d77b8e: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:32.601547 4952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a29dd7681fcc4ef0b7cc21e7a4d77b8e: No bootstrap required, opened a new log
I20260811 05:17:32.605019 4952 raft_consensus.cc:348] T 00000000000000000000000000000000 P a29dd7681fcc4ef0b7cc21e7a4d77b8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29dd7681fcc4ef0b7cc21e7a4d77b8e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 44655 } }
I20260811 05:17:32.605265 4952 raft_consensus.cc:374] T 00000000000000000000000000000000 P a29dd7681fcc4ef0b7cc21e7a4d77b8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:32.605333 4952 raft_consensus.cc:729] T 00000000000000000000000000000000 P a29dd7681fcc4ef0b7cc21e7a4d77b8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a29dd7681fcc4ef0b7cc21e7a4d77b8e, State: Initialized, Role: FOLLOWER
I20260811 05:17:32.605810 4952 consensus_queue.cc:260] T 00000000000000000000000000000000 P a29dd7681fcc4ef0b7cc21e7a4d77b8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29dd7681fcc4ef0b7cc21e7a4d77b8e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 44655 } }
I20260811 05:17:32.605989 4952 raft_consensus.cc:388] T 00000000000000000000000000000000 P a29dd7681fcc4ef0b7cc21e7a4d77b8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:32.606068 4952 raft_consensus.cc:482] T 00000000000000000000000000000000 P a29dd7681fcc4ef0b7cc21e7a4d77b8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:32.606187 4952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a29dd7681fcc4ef0b7cc21e7a4d77b8e [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:32.606513 4957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:32.598173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55306 (local address 127.7.67.190:44655)
0811 05:17:32.598905 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:32.598921 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:32.598959 (+ 38us) server_negotiation.cc:407] Connection header received
0811 05:17:32.599634 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:32.599664 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:32.600177 (+ 513us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:32.600582 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:32.601033 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:32.602069 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:32.602980 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:32.603463 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:32.603633 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:32.604505 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:32.604511 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:32.604561 (+ 50us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:32.604972 (+ 411us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:32.605028 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:32.605048 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:32.605161 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:32.605295 (+ 134us) server_negotiation.cc:299] Negotiation successful
0811 05:17:32.605575 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":148,"threads_started":1}
I20260811 05:17:32.607252 4952 raft_consensus.cc:504] T 00000000000000000000000000000000 P a29dd7681fcc4ef0b7cc21e7a4d77b8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29dd7681fcc4ef0b7cc21e7a4d77b8e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 44655 } }
I20260811 05:17:32.607605 4952 leader_election.cc:302] T 00000000000000000000000000000000 P a29dd7681fcc4ef0b7cc21e7a4d77b8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a29dd7681fcc4ef0b7cc21e7a4d77b8e; no voters:
I20260811 05:17:32.607957 4952 leader_election.cc:288] T 00000000000000000000000000000000 P a29dd7681fcc4ef0b7cc21e7a4d77b8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:32.608026 4959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a29dd7681fcc4ef0b7cc21e7a4d77b8e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:32.608253 4959 raft_consensus.cc:686] T 00000000000000000000000000000000 P a29dd7681fcc4ef0b7cc21e7a4d77b8e [term 1 LEADER]: Becoming Leader. State: Replica: a29dd7681fcc4ef0b7cc21e7a4d77b8e, State: Running, Role: LEADER
I20260811 05:17:32.608579 4959 consensus_queue.cc:237] T 00000000000000000000000000000000 P a29dd7681fcc4ef0b7cc21e7a4d77b8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29dd7681fcc4ef0b7cc21e7a4d77b8e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 44655 } }
I20260811 05:17:32.609293 4952 sys_catalog.cc:564] T 00000000000000000000000000000000 P a29dd7681fcc4ef0b7cc21e7a4d77b8e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:32.609797 4959 sys_catalog.cc:455] T 00000000000000000000000000000000 P a29dd7681fcc4ef0b7cc21e7a4d77b8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a29dd7681fcc4ef0b7cc21e7a4d77b8e. Latest consensus state: current_term: 1 leader_uuid: "a29dd7681fcc4ef0b7cc21e7a4d77b8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29dd7681fcc4ef0b7cc21e7a4d77b8e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 44655 } } }
I20260811 05:17:32.609916 4959 sys_catalog.cc:458] T 00000000000000000000000000000000 P a29dd7681fcc4ef0b7cc21e7a4d77b8e [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:32.610294 4960 sys_catalog.cc:455] T 00000000000000000000000000000000 P a29dd7681fcc4ef0b7cc21e7a4d77b8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a29dd7681fcc4ef0b7cc21e7a4d77b8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a29dd7681fcc4ef0b7cc21e7a4d77b8e" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 44655 } } }
I20260811 05:17:32.610462 4960 sys_catalog.cc:458] T 00000000000000000000000000000000 P a29dd7681fcc4ef0b7cc21e7a4d77b8e [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:32.611670 4966 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:32.614339 4966 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:32.619745 4966 catalog_manager.cc:1347] Generated new cluster ID: 425934481ea041ae84ba0533617ccceb
I20260811 05:17:32.619820 4966 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:32.631850 4966 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:32.633068 4966 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:32.638825 4966 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a29dd7681fcc4ef0b7cc21e7a4d77b8e: Generated new TSK 0
I20260811 05:17:32.639482 4966 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:32.649873 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:44655
--builtin_ntp_servers=127.7.67.148:44843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 05:17:32.777591 4993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:32.777854 4993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:32.777894 4993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:32.782194 4993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:32.782253 4993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:32.782307 4993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:17:32.787190 4993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:44843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:44655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:32.788298 4993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:32.789691 4993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:32.799487 5000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:32.799474 5005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:32.799710 4993 server_base.cc:1034] running on GCE node
W20260811 05:17:32.799490 5001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:32.800175 4993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:32.800889 4993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:32.802047 4993 hybrid_clock.cc:648] HybridClock initialized: now 1786425452802030 us; error 28 us; skew 500 ppm
I20260811 05:17:32.804009 4993 webserver.cc:460] Webserver started at http://127.7.67.129:37711/ using document root <none> and password file <none>
I20260811 05:17:32.804659 4993 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:32.804730 4993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:32.804973 4993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:32.807075 4993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "7f28e17f10754e3482b41eeb08be63ed"
format_stamp: "Formatted at 2026-08-11 05:17:32 on dist-test-slave-04bb"
server_key: "3c82bd0aadef01cc6a00b83c694bdb2b"
server_key_iv: "573c986fe4f10a041a8a937ebc708fb8"
server_key_version: "encryptionkey@0"
I20260811 05:17:32.807636 4993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "7f28e17f10754e3482b41eeb08be63ed"
format_stamp: "Formatted at 2026-08-11 05:17:32 on dist-test-slave-04bb"
server_key: "3c82bd0aadef01cc6a00b83c694bdb2b"
server_key_iv: "573c986fe4f10a041a8a937ebc708fb8"
server_key_version: "encryptionkey@0"
I20260811 05:17:32.812148 4993 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260811 05:17:32.814729 5014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:32.816085 4993 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:17:32.816211 4993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "7f28e17f10754e3482b41eeb08be63ed"
format_stamp: "Formatted at 2026-08-11 05:17:32 on dist-test-slave-04bb"
server_key: "3c82bd0aadef01cc6a00b83c694bdb2b"
server_key_iv: "573c986fe4f10a041a8a937ebc708fb8"
server_key_version: "encryptionkey@0"
I20260811 05:17:32.816340 4993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:32.828460 4993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:32.829170 4993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:32.829370 4993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:32.829973 4993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:32.830935 4993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:32.831005 4993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:32.831079 4993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:32.831118 4993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:32.839902 4993 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:40995
I20260811 05:17:32.839946 5195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:40995 every 8 connection(s)
I20260811 05:17:32.840862 4993 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:17:32.847790 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 4993
I20260811 05:17:32.847910 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:17:32.848281 7438 external_mini_cluster.cc:1428] Setting key 16a8972087c52be6402a92164361f101
I20260811 05:17:32.850589 4957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:32.842773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:60555 (local address 127.7.67.190:44655)
0811 05:17:32.842911 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:32.842915 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:32.843993 (+ 1078us) server_negotiation.cc:407] Connection header received
0811 05:17:32.845414 (+ 1421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:32.845417 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:32.845466 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:32.845546 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:32.846440 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:32.846970 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:32.848290 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:32.848633 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:32.848760 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:32.850002 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:32.850006 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:32.850008 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:32.850023 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:32.850045 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:32.850053 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:32.850095 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:32.850333 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 05:17:32.850411 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 05:17:32.850590 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:44655
--builtin_ntp_servers=127.7.67.148:44843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 05:17:32.851493 5202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:32.843155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:44655 (local address 127.7.67.129:60555)
0811 05:17:32.843830 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:32.843871 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:32.845180 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:32.845718 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:32.845724 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:32.845805 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:32.846300 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:32.846312 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:32.847084 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:32.847088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:32.848146 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:32.848158 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:32.848824 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:32.848828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:32.849021 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:32.849699 (+ 678us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:17:32.849701 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:32.849753 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:17:32.849832 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:32.850153 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:32.850162 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:32.850167 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 05:17:32.850282 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 05:17:32.850588 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":128,"threads_started":1}
I20260811 05:17:32.852855 5197 heartbeater.cc:347] Connected to a master server at 127.7.67.190:44655
I20260811 05:17:32.853091 5197 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:32.853609 5197 heartbeater.cc:511] Master 127.7.67.190:44655 requested a full tablet report, sending...
I20260811 05:17:32.855075 4890 ts_manager.cc:194] Registered new tserver with Master: 7f28e17f10754e3482b41eeb08be63ed (127.7.67.129:40995)
I20260811 05:17:32.856208 4890 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.67.129:60555
W20260811 05:17:32.970850 5203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:32.971104 5203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:32.971140 5203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:32.975106 5203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:32.975162 5203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:32.975214 5203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:17:32.979987 5203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:44843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:44655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:32.981189 5203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:32.982712 5203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:32.992483 5214 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:32.992660 5203 server_base.cc:1034] running on GCE node
W20260811 05:17:32.992858 5220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:32.992885 5216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:32.993417 5203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:32.994019 5203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:32.995177 5203 hybrid_clock.cc:648] HybridClock initialized: now 1786425452995159 us; error 30 us; skew 500 ppm
I20260811 05:17:32.997117 5203 webserver.cc:460] Webserver started at http://127.7.67.130:45067/ using document root <none> and password file <none>
I20260811 05:17:32.997691 5203 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:32.997800 5203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:32.998034 5203 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:33.000012 5203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "0f79c3aafad84a6594e4245fe87c9c21"
format_stamp: "Formatted at 2026-08-11 05:17:32 on dist-test-slave-04bb"
server_key: "af9eed5d06cd414db35c7f1bdc01549f"
server_key_iv: "0c1ff0286a69d40957f4de548a785146"
server_key_version: "encryptionkey@0"
I20260811 05:17:33.000623 5203 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "0f79c3aafad84a6594e4245fe87c9c21"
format_stamp: "Formatted at 2026-08-11 05:17:32 on dist-test-slave-04bb"
server_key: "af9eed5d06cd414db35c7f1bdc01549f"
server_key_iv: "0c1ff0286a69d40957f4de548a785146"
server_key_version: "encryptionkey@0"
I20260811 05:17:33.004163 5203 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 05:17:33.006574 5233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:33.007705 5203 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:17:33.007805 5203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "0f79c3aafad84a6594e4245fe87c9c21"
format_stamp: "Formatted at 2026-08-11 05:17:32 on dist-test-slave-04bb"
server_key: "af9eed5d06cd414db35c7f1bdc01549f"
server_key_iv: "0c1ff0286a69d40957f4de548a785146"
server_key_version: "encryptionkey@0"
I20260811 05:17:33.007929 5203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:33.022614 5203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:33.023219 5203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:33.023361 5203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:33.023873 5203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:33.024871 5203 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:33.024920 5203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:33.024961 5203 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:33.024983 5203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:33.033720 5203 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:41869
I20260811 05:17:33.033763 5435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:41869 every 8 connection(s)
I20260811 05:17:33.034615 5203 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:17:33.038727 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 5203
I20260811 05:17:33.038878 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:17:33.039220 7438 external_mini_cluster.cc:1428] Setting key 85b4c7772ce76b6799765531f62b7eb5
I20260811 05:17:33.041345 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:44655
--builtin_ntp_servers=127.7.67.148:44843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 05:17:33.044221 4957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:33.036420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:55647 (local address 127.7.67.190:44655)
0811 05:17:33.036614 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:33.036619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:33.037633 (+ 1014us) server_negotiation.cc:407] Connection header received
0811 05:17:33.039319 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:33.039325 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:33.039385 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:33.039459 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:33.040583 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.041129 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.042340 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.042556 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.042666 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:33.043736 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:33.043740 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:33.043743 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:33.043762 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:33.043782 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:33.043790 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:33.043865 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:33.044016 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 05:17:33.044078 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 05:17:33.045230 5443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:33.036846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:44655 (local address 127.7.67.130:55647)
0811 05:17:33.037483 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:33.037524 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:33.039044 (+ 1520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:33.039676 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:33.039685 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:33.039780 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:33.040388 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:33.040407 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.041255 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.041263 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:33.042201 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:33.042211 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.042705 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.042709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:33.042879 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:33.043484 (+ 605us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:17:33.043485 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:33.043524 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:17:33.043585 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:33.043869 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:33.043880 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:33.043888 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 05:17:33.044015 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 05:17:33.044280 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":120,"threads_started":1}
I20260811 05:17:33.046351 5439 heartbeater.cc:347] Connected to a master server at 127.7.67.190:44655
I20260811 05:17:33.046620 5439 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:33.047324 5439 heartbeater.cc:511] Master 127.7.67.190:44655 requested a full tablet report, sending...
I20260811 05:17:33.048516 4890 ts_manager.cc:194] Registered new tserver with Master: 0f79c3aafad84a6594e4245fe87c9c21 (127.7.67.130:41869)
I20260811 05:17:33.049422 4890 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.67.130:55647
W20260811 05:17:33.159816 5444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:33.160053 5444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:33.160089 5444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:33.164048 5444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:33.164134 5444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:33.164192 5444 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:17:33.168948 5444 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:44843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:44655
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:33.170064 5444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:33.171487 5444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:33.182081 5459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:33.182169 5454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:33.182490 5444 server_base.cc:1034] running on GCE node
W20260811 05:17:33.182054 5461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:33.183007 5444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:33.183661 5444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:33.184819 5444 hybrid_clock.cc:648] HybridClock initialized: now 1786425453184787 us; error 44 us; skew 500 ppm
I20260811 05:17:33.186725 5444 webserver.cc:460] Webserver started at http://127.7.67.131:45879/ using document root <none> and password file <none>
I20260811 05:17:33.187333 5444 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:33.187397 5444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:33.187610 5444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:33.189805 5444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "ee1479c8a07544e3ae9dc09c9ff81ee6"
format_stamp: "Formatted at 2026-08-11 05:17:33 on dist-test-slave-04bb"
server_key: "e8a1c73af76483fe6b08a7b78cbf5b72"
server_key_iv: "4b25af652d51593c259a7cffdae4f850"
server_key_version: "encryptionkey@0"
I20260811 05:17:33.190436 5444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "ee1479c8a07544e3ae9dc09c9ff81ee6"
format_stamp: "Formatted at 2026-08-11 05:17:33 on dist-test-slave-04bb"
server_key: "e8a1c73af76483fe6b08a7b78cbf5b72"
server_key_iv: "4b25af652d51593c259a7cffdae4f850"
server_key_version: "encryptionkey@0"
I20260811 05:17:33.194199 5444 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 05:17:33.196947 5470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:33.198235 5444 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:17:33.198378 5444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "ee1479c8a07544e3ae9dc09c9ff81ee6"
format_stamp: "Formatted at 2026-08-11 05:17:33 on dist-test-slave-04bb"
server_key: "e8a1c73af76483fe6b08a7b78cbf5b72"
server_key_iv: "4b25af652d51593c259a7cffdae4f850"
server_key_version: "encryptionkey@0"
I20260811 05:17:33.198514 5444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:33.222251 5444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:33.223016 5444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:33.223222 5444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:33.223860 5444 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:33.224907 5444 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:33.224980 5444 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:33.225050 5444 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:33.225098 5444 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:33.234226 5444 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:39473
I20260811 05:17:33.234258 5657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:39473 every 8 connection(s)
I20260811 05:17:33.235189 5444 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:17:33.239329 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 5444
I20260811 05:17:33.239444 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:17:33.239740 7438 external_mini_cluster.cc:1428] Setting key c28bed10dd4ea9d441228d9da6957158
I20260811 05:17:33.244928 4957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:33.237206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:41903 (local address 127.7.67.190:44655)
0811 05:17:33.237361 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:33.237366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:33.238495 (+ 1129us) server_negotiation.cc:407] Connection header received
0811 05:17:33.240017 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:33.240020 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:33.240087 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:33.240182 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:33.241315 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.241906 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.243051 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.243298 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.243382 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:33.244436 (+ 1054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:33.244439 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:33.244441 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:33.244452 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:33.244468 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:33.244475 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:33.244590 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:33.244767 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 05:17:33.244817 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 05:17:33.245877 5663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:33.237670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:44655 (local address 127.7.67.131:41903)
0811 05:17:33.238344 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:33.238387 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:33.239753 (+ 1366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:33.240405 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:33.240413 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:33.240502 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:33.241169 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:33.241192 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.242035 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.242042 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:33.242937 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:33.242946 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.243428 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.243431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:33.243564 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:33.244198 (+ 634us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:17:33.244200 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:33.244240 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:17:33.244309 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:33.244602 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:33.244612 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:33.244618 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 05:17:33.244745 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 05:17:33.244983 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":116,"threads_started":1}
I20260811 05:17:33.246991 5659 heartbeater.cc:347] Connected to a master server at 127.7.67.190:44655
I20260811 05:17:33.247279 5659 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:33.247812 5659 heartbeater.cc:511] Master 127.7.67.190:44655 requested a full tablet report, sending...
I20260811 05:17:33.248878 4890 ts_manager.cc:194] Registered new tserver with Master: ee1479c8a07544e3ae9dc09c9ff81ee6 (127.7.67.131:39473)
I20260811 05:17:33.249413 4890 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.67.131:41903
I20260811 05:17:33.253729 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:17:33.259990 4957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:33.255719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55312 (local address 127.7.67.190:44655)
0811 05:17:33.255838 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:33.255842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:33.256012 (+ 170us) server_negotiation.cc:407] Connection header received
0811 05:17:33.256112 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:33.256114 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:33.256152 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:33.256220 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:33.256487 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.257061 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.257865 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.258107 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.258225 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:33.259392 (+ 1167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:33.259396 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:33.259399 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:33.259413 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:33.259434 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:33.259446 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:33.259590 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:33.259807 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 05:17:33.259847 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 05:17:33.263126 4890 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:55312:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:17:33.266335 4890 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:17:33.281322 5683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:33.276046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:41869 (local address 127.0.0.1:34224)
0811 05:17:33.276815 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:33.276829 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:33.276942 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:33.277378 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:33.277381 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:33.277399 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:33.277476 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:33.277482 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.278461 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.278464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:33.279931 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:33.279938 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.280782 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.280786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:33.280888 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:33.280924 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 05:17:33.281038 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 05:17:33.281166 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"mutex_wait_us":114,"thread_start_us":98,"threads_started":1}
I20260811 05:17:33.281335 5684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:33.276278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34224 (local address 127.7.67.130:41869)
0811 05:17:33.276693 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:33.276705 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:33.277040 (+ 335us) server_negotiation.cc:407] Connection header received
0811 05:17:33.277107 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:33.277111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:33.277250 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:33.277403 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:33.277591 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.278352 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.280045 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.280663 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.280782 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:33.280985 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:33.281067 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 05:17:33.281162 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":96,"threads_started":1}
I20260811 05:17:33.281724 5678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:33.274962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:39473 (local address 127.0.0.1:57126)
0811 05:17:33.275577 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:33.275613 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:33.275758 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:33.276778 (+ 1020us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:33.276781 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:33.276805 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:33.276903 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:33.276909 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.278365 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.278374 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:33.279822 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:33.279833 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.281067 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.281071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:33.281301 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:33.281407 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 05:17:33.281487 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 05:17:33.281568 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"mutex_wait_us":203,"spinlock_wait_cycles":7168,"thread_start_us":131,"threads_started":1}
I20260811 05:17:33.281802 5679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:33.274901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:40995 (local address 127.0.0.1:47042)
0811 05:17:33.275563 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:33.275613 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:33.275758 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:33.276647 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:33.276656 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:33.276683 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:33.276839 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:33.276851 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.278334 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.278337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:33.279747 (+ 1410us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:33.279758 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.281067 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.281071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:33.281301 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:33.281560 (+ 259us) client_negotiation.cc:769] Sending connection context
0811 05:17:33.281621 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 05:17:33.281677 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"mutex_wait_us":353,"spinlock_wait_cycles":3712,"thread_start_us":76,"threads_started":1}
I20260811 05:17:33.282194 5681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:33.275721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57126 (local address 127.7.67.131:39473)
0811 05:17:33.276069 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:33.276081 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:33.276106 (+ 25us) server_negotiation.cc:407] Connection header received
0811 05:17:33.276184 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:33.276192 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:33.276388 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:33.276573 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:33.277048 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.278215 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.279929 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.280697 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.280811 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:33.280996 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:33.281926 (+ 930us) server_negotiation.cc:299] Negotiation successful
0811 05:17:33.282034 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":157,"threads_started":1}
I20260811 05:17:33.282557 5680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:33.275026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47042 (local address 127.7.67.129:40995)
0811 05:17:33.275366 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:33.275378 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:33.275905 (+ 527us) server_negotiation.cc:407] Connection header received
0811 05:17:33.276011 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:33.276017 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:33.276243 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:33.276424 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:33.276982 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.278206 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.279871 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.280697 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.280819 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:33.280996 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:33.282012 (+ 1016us) server_negotiation.cc:299] Negotiation successful
0811 05:17:33.282101 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":119,"threads_started":1}
I20260811 05:17:33.285044 5085 tablet_service.cc:1467] Processing CreateTablet for tablet d78afd01a077465887c51e78b65ab1db (DEFAULT_TABLE table=test-table [id=470ed3e3dfdf48a7a640a2f5b47a57db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:33.285059 5328 tablet_service.cc:1467] Processing CreateTablet for tablet d78afd01a077465887c51e78b65ab1db (DEFAULT_TABLE table=test-table [id=470ed3e3dfdf48a7a640a2f5b47a57db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:33.285059 5557 tablet_service.cc:1467] Processing CreateTablet for tablet d78afd01a077465887c51e78b65ab1db (DEFAULT_TABLE table=test-table [id=470ed3e3dfdf48a7a640a2f5b47a57db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:33.286067 5085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d78afd01a077465887c51e78b65ab1db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:33.286095 5557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d78afd01a077465887c51e78b65ab1db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:33.286088 5328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d78afd01a077465887c51e78b65ab1db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:33.292589 5686 tablet_bootstrap.cc:492] T d78afd01a077465887c51e78b65ab1db P 7f28e17f10754e3482b41eeb08be63ed: Bootstrap starting.
I20260811 05:17:33.293939 5690 tablet_bootstrap.cc:492] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21: Bootstrap starting.
I20260811 05:17:33.294785 5686 tablet_bootstrap.cc:654] T d78afd01a077465887c51e78b65ab1db P 7f28e17f10754e3482b41eeb08be63ed: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:33.295526 5686 log.cc:824] T d78afd01a077465887c51e78b65ab1db P 7f28e17f10754e3482b41eeb08be63ed: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:33.295719 5690 tablet_bootstrap.cc:654] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:33.296197 5687 tablet_bootstrap.cc:492] T d78afd01a077465887c51e78b65ab1db P ee1479c8a07544e3ae9dc09c9ff81ee6: Bootstrap starting.
I20260811 05:17:33.296581 5690 log.cc:824] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:33.297768 5686 tablet_bootstrap.cc:492] T d78afd01a077465887c51e78b65ab1db P 7f28e17f10754e3482b41eeb08be63ed: No bootstrap required, opened a new log
I20260811 05:17:33.297978 5686 ts_tablet_manager.cc:1397] T d78afd01a077465887c51e78b65ab1db P 7f28e17f10754e3482b41eeb08be63ed: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 05:17:33.298363 5687 tablet_bootstrap.cc:654] T d78afd01a077465887c51e78b65ab1db P ee1479c8a07544e3ae9dc09c9ff81ee6: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:33.298993 5687 log.cc:824] T d78afd01a077465887c51e78b65ab1db P ee1479c8a07544e3ae9dc09c9ff81ee6: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:33.299137 5690 tablet_bootstrap.cc:492] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21: No bootstrap required, opened a new log
I20260811 05:17:33.299324 5690 ts_tablet_manager.cc:1397] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260811 05:17:33.300640 5687 tablet_bootstrap.cc:492] T d78afd01a077465887c51e78b65ab1db P ee1479c8a07544e3ae9dc09c9ff81ee6: No bootstrap required, opened a new log
I20260811 05:17:33.300832 5687 ts_tablet_manager.cc:1397] T d78afd01a077465887c51e78b65ab1db P ee1479c8a07544e3ae9dc09c9ff81ee6: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:17:33.302419 5686 raft_consensus.cc:348] T d78afd01a077465887c51e78b65ab1db P 7f28e17f10754e3482b41eeb08be63ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f28e17f10754e3482b41eeb08be63ed" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 40995 } } peers { permanent_uuid: "ee1479c8a07544e3ae9dc09c9ff81ee6" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39473 } } peers { permanent_uuid: "0f79c3aafad84a6594e4245fe87c9c21" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 41869 } }
I20260811 05:17:33.302685 5686 raft_consensus.cc:374] T d78afd01a077465887c51e78b65ab1db P 7f28e17f10754e3482b41eeb08be63ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:33.302735 5686 raft_consensus.cc:729] T d78afd01a077465887c51e78b65ab1db P 7f28e17f10754e3482b41eeb08be63ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f28e17f10754e3482b41eeb08be63ed, State: Initialized, Role: FOLLOWER
I20260811 05:17:33.303267 5686 consensus_queue.cc:260] T d78afd01a077465887c51e78b65ab1db P 7f28e17f10754e3482b41eeb08be63ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f28e17f10754e3482b41eeb08be63ed" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 40995 } } peers { permanent_uuid: "ee1479c8a07544e3ae9dc09c9ff81ee6" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39473 } } peers { permanent_uuid: "0f79c3aafad84a6594e4245fe87c9c21" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 41869 } }
I20260811 05:17:33.303565 5690 raft_consensus.cc:348] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f28e17f10754e3482b41eeb08be63ed" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 40995 } } peers { permanent_uuid: "ee1479c8a07544e3ae9dc09c9ff81ee6" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39473 } } peers { permanent_uuid: "0f79c3aafad84a6594e4245fe87c9c21" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 41869 } }
I20260811 05:17:33.303825 5690 raft_consensus.cc:374] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:33.304080 5197 heartbeater.cc:503] Master 127.7.67.190:44655 was elected leader, sending a full tablet report...
I20260811 05:17:33.304332 5686 ts_tablet_manager.cc:1428] T d78afd01a077465887c51e78b65ab1db P 7f28e17f10754e3482b41eeb08be63ed: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 05:17:33.304780 5690 raft_consensus.cc:729] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f79c3aafad84a6594e4245fe87c9c21, State: Initialized, Role: FOLLOWER
I20260811 05:17:33.305083 5687 raft_consensus.cc:348] T d78afd01a077465887c51e78b65ab1db P ee1479c8a07544e3ae9dc09c9ff81ee6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f28e17f10754e3482b41eeb08be63ed" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 40995 } } peers { permanent_uuid: "ee1479c8a07544e3ae9dc09c9ff81ee6" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39473 } } peers { permanent_uuid: "0f79c3aafad84a6594e4245fe87c9c21" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 41869 } }
I20260811 05:17:33.305207 5690 consensus_queue.cc:260] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f28e17f10754e3482b41eeb08be63ed" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 40995 } } peers { permanent_uuid: "ee1479c8a07544e3ae9dc09c9ff81ee6" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39473 } } peers { permanent_uuid: "0f79c3aafad84a6594e4245fe87c9c21" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 41869 } }
I20260811 05:17:33.305408 5687 raft_consensus.cc:374] T d78afd01a077465887c51e78b65ab1db P ee1479c8a07544e3ae9dc09c9ff81ee6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:33.305508 5687 raft_consensus.cc:729] T d78afd01a077465887c51e78b65ab1db P ee1479c8a07544e3ae9dc09c9ff81ee6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee1479c8a07544e3ae9dc09c9ff81ee6, State: Initialized, Role: FOLLOWER
I20260811 05:17:33.305927 5439 heartbeater.cc:503] Master 127.7.67.190:44655 was elected leader, sending a full tablet report...
I20260811 05:17:33.305982 5687 consensus_queue.cc:260] T d78afd01a077465887c51e78b65ab1db P ee1479c8a07544e3ae9dc09c9ff81ee6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f28e17f10754e3482b41eeb08be63ed" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 40995 } } peers { permanent_uuid: "ee1479c8a07544e3ae9dc09c9ff81ee6" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39473 } } peers { permanent_uuid: "0f79c3aafad84a6594e4245fe87c9c21" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 41869 } }
I20260811 05:17:33.306123 5690 ts_tablet_manager.cc:1428] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21: Time spent starting tablet: real 0.007s user 0.002s sys 0.005s
I20260811 05:17:33.306747 5687 ts_tablet_manager.cc:1428] T d78afd01a077465887c51e78b65ab1db P ee1479c8a07544e3ae9dc09c9ff81ee6: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 05:17:33.307109 5659 heartbeater.cc:503] Master 127.7.67.190:44655 was elected leader, sending a full tablet report...
I20260811 05:17:33.316690 5699 raft_consensus.cc:482] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:17:33.316840 5699 raft_consensus.cc:504] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f28e17f10754e3482b41eeb08be63ed" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 40995 } } peers { permanent_uuid: "ee1479c8a07544e3ae9dc09c9ff81ee6" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39473 } } peers { permanent_uuid: "0f79c3aafad84a6594e4245fe87c9c21" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 41869 } }
I20260811 05:17:33.317828 5699 leader_election.cc:288] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f28e17f10754e3482b41eeb08be63ed (127.7.67.129:40995), ee1479c8a07544e3ae9dc09c9ff81ee6 (127.7.67.131:39473)
I20260811 05:17:33.322135 5681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:33.318159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:32771 (local address 127.7.67.131:39473)
0811 05:17:33.318294 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:33.318303 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:33.318317 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:17:33.318386 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:33.318389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:33.318427 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:33.318491 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:33.318848 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.319573 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.321040 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.321605 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.321695 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:33.321735 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:33.321983 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 05:17:33.322035 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:17:33.322203 5443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:33.318047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:39473 (local address 127.7.67.130:32771)
0811 05:17:33.318196 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:33.318215 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:33.318309 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:33.318535 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:33.318541 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:33.318567 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:33.318724 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:33.318732 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.319679 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.319683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:33.320914 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:33.320931 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.321725 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.321730 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:33.321845 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:33.321874 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 05:17:33.321970 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 05:17:33.322042 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260811 05:17:33.322921 5702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:33.318048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:40995 (local address 127.7.67.130:53885)
0811 05:17:33.318758 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:33.318770 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:33.318906 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:33.319143 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:33.319146 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:33.319164 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:33.319287 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:33.319293 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.320314 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.320317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:33.321380 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:33.321390 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.322517 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.322521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:33.322590 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:33.322634 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 05:17:33.322713 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 05:17:33.322778 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"mutex_wait_us":330,"thread_start_us":186,"threads_started":1}
I20260811 05:17:33.322923 5591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d78afd01a077465887c51e78b65ab1db" candidate_uuid: "0f79c3aafad84a6594e4245fe87c9c21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee1479c8a07544e3ae9dc09c9ff81ee6" is_pre_election: true
I20260811 05:17:33.323140 5680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:33.318095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:53885 (local address 127.7.67.129:40995)
0811 05:17:33.318240 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:33.318245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:33.318877 (+ 632us) server_negotiation.cc:407] Connection header received
0811 05:17:33.319001 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:33.319003 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:33.319047 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:33.319131 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:33.319399 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.320118 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.321524 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.322155 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.322292 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:33.322343 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:33.322937 (+ 594us) server_negotiation.cc:299] Negotiation successful
0811 05:17:33.323019 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 05:17:33.323231 5591 raft_consensus.cc:2454] T d78afd01a077465887c51e78b65ab1db P ee1479c8a07544e3ae9dc09c9ff81ee6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f79c3aafad84a6594e4245fe87c9c21 in term 0.
I20260811 05:17:33.323755 5234 leader_election.cc:302] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f79c3aafad84a6594e4245fe87c9c21, ee1479c8a07544e3ae9dc09c9ff81ee6; no voters:
I20260811 05:17:33.323961 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d78afd01a077465887c51e78b65ab1db" candidate_uuid: "0f79c3aafad84a6594e4245fe87c9c21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f28e17f10754e3482b41eeb08be63ed" is_pre_election: true
I20260811 05:17:33.324054 5699 raft_consensus.cc:2781] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:17:33.324162 5699 raft_consensus.cc:482] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:17:33.324218 5699 raft_consensus.cc:3037] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:33.324303 5116 raft_consensus.cc:2454] T d78afd01a077465887c51e78b65ab1db P 7f28e17f10754e3482b41eeb08be63ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f79c3aafad84a6594e4245fe87c9c21 in term 0.
I20260811 05:17:33.325393 5699 raft_consensus.cc:504] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f28e17f10754e3482b41eeb08be63ed" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 40995 } } peers { permanent_uuid: "ee1479c8a07544e3ae9dc09c9ff81ee6" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39473 } } peers { permanent_uuid: "0f79c3aafad84a6594e4245fe87c9c21" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 41869 } }
I20260811 05:17:33.325733 5699 leader_election.cc:288] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21 [CANDIDATE]: Term 1 election: Requested vote from peers 7f28e17f10754e3482b41eeb08be63ed (127.7.67.129:40995), ee1479c8a07544e3ae9dc09c9ff81ee6 (127.7.67.131:39473)
I20260811 05:17:33.326103 5591 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d78afd01a077465887c51e78b65ab1db" candidate_uuid: "0f79c3aafad84a6594e4245fe87c9c21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee1479c8a07544e3ae9dc09c9ff81ee6"
I20260811 05:17:33.326236 5591 raft_consensus.cc:3037] T d78afd01a077465887c51e78b65ab1db P ee1479c8a07544e3ae9dc09c9ff81ee6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:33.327179 5591 raft_consensus.cc:2454] T d78afd01a077465887c51e78b65ab1db P ee1479c8a07544e3ae9dc09c9ff81ee6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f79c3aafad84a6594e4245fe87c9c21 in term 1.
I20260811 05:17:33.327478 5234 leader_election.cc:302] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f79c3aafad84a6594e4245fe87c9c21, ee1479c8a07544e3ae9dc09c9ff81ee6; no voters:
I20260811 05:17:33.327461 5116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d78afd01a077465887c51e78b65ab1db" candidate_uuid: "0f79c3aafad84a6594e4245fe87c9c21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f28e17f10754e3482b41eeb08be63ed"
I20260811 05:17:33.327615 5116 raft_consensus.cc:3037] T d78afd01a077465887c51e78b65ab1db P 7f28e17f10754e3482b41eeb08be63ed [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:33.327684 5699 raft_consensus.cc:2781] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:33.327965 5699 raft_consensus.cc:686] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21 [term 1 LEADER]: Becoming Leader. State: Replica: 0f79c3aafad84a6594e4245fe87c9c21, State: Running, Role: LEADER
I20260811 05:17:33.328320 5699 consensus_queue.cc:237] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f28e17f10754e3482b41eeb08be63ed" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 40995 } } peers { permanent_uuid: "ee1479c8a07544e3ae9dc09c9ff81ee6" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39473 } } peers { permanent_uuid: "0f79c3aafad84a6594e4245fe87c9c21" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 41869 } }
I20260811 05:17:33.328919 5116 raft_consensus.cc:2454] T d78afd01a077465887c51e78b65ab1db P 7f28e17f10754e3482b41eeb08be63ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f79c3aafad84a6594e4245fe87c9c21 in term 1.
I20260811 05:17:33.331817 4890 catalog_manager.cc:5495] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f79c3aafad84a6594e4245fe87c9c21 (127.7.67.130). New cstate: current_term: 1 leader_uuid: "0f79c3aafad84a6594e4245fe87c9c21" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f28e17f10754e3482b41eeb08be63ed" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 40995 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee1479c8a07544e3ae9dc09c9ff81ee6" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39473 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f79c3aafad84a6594e4245fe87c9c21" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 41869 } health_report { overall_health: HEALTHY } } }
W20260811 05:17:33.342111 5201 tablet.cc:2369] T d78afd01a077465887c51e78b65ab1db P 7f28e17f10754e3482b41eeb08be63ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:17:33.344826 5684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:33.341240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34226 (local address 127.7.67.130:41869)
0811 05:17:33.341374 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:33.341378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:33.341391 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:17:33.341452 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:33.341454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:33.341505 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:33.341592 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:17:33.341852 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.342600 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.343477 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:33.343674 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:33.343747 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:33.344009 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:17:33.344392 (+ 383us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:17:33.344500 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:33.344622 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 05:17:33.344654 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 05:17:33.351037 5591 raft_consensus.cc:1261] T d78afd01a077465887c51e78b65ab1db P ee1479c8a07544e3ae9dc09c9ff81ee6 [term 1 FOLLOWER]: Refusing update from remote peer 0f79c3aafad84a6594e4245fe87c9c21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:33.351044 5116 raft_consensus.cc:1261] T d78afd01a077465887c51e78b65ab1db P 7f28e17f10754e3482b41eeb08be63ed [term 1 FOLLOWER]: Refusing update from remote peer 0f79c3aafad84a6594e4245fe87c9c21: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:33.351745 5704 consensus_queue.cc:1035] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f28e17f10754e3482b41eeb08be63ed" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 40995 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:33.351943 5699 consensus_queue.cc:1035] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee1479c8a07544e3ae9dc09c9ff81ee6" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 39473 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:33.359122 5709 mvcc.cc:205] Tried to move back new op lower bound from 7317198656917692416 to 7317198656836120576. Current Snapshot: MvccSnapshot[applied={T|T < 7317198656917692416}]
I20260811 05:17:33.360589 5710 mvcc.cc:205] Tried to move back new op lower bound from 7317198656917692416 to 7317198656836120576. Current Snapshot: MvccSnapshot[applied={T|T < 7317198656917692416}]
I20260811 05:17:33.369270 4885 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:55312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:17:33.369458 4885 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:55312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:17:33.371909 4885 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a29dd7681fcc4ef0b7cc21e7a4d77b8e: Sending DeleteTablet for 3 replicas of tablet d78afd01a077465887c51e78b65ab1db
I20260811 05:17:33.372740 5085 tablet_service.cc:1514] Processing DeleteTablet for tablet d78afd01a077465887c51e78b65ab1db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:33 UTC) from {username='slave'} at 127.0.0.1:47042
I20260811 05:17:33.372843 5557 tablet_service.cc:1514] Processing DeleteTablet for tablet d78afd01a077465887c51e78b65ab1db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:33 UTC) from {username='slave'} at 127.0.0.1:57126
I20260811 05:17:33.372954 5328 tablet_service.cc:1514] Processing DeleteTablet for tablet d78afd01a077465887c51e78b65ab1db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:33 UTC) from {username='slave'} at 127.0.0.1:34224
I20260811 05:17:33.373430 5722 tablet_replica.cc:330] T d78afd01a077465887c51e78b65ab1db P 7f28e17f10754e3482b41eeb08be63ed: stopping tablet replica
I20260811 05:17:33.373510 5723 tablet_replica.cc:330] T d78afd01a077465887c51e78b65ab1db P ee1479c8a07544e3ae9dc09c9ff81ee6: stopping tablet replica
I20260811 05:17:33.373845 5723 raft_consensus.cc:2229] T d78afd01a077465887c51e78b65ab1db P ee1479c8a07544e3ae9dc09c9ff81ee6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:17:33.373982 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 4993
I20260811 05:17:33.374178 5723 raft_consensus.cc:2258] T d78afd01a077465887c51e78b65ab1db P ee1479c8a07544e3ae9dc09c9ff81ee6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:33.379681 5725 tablet_replica.cc:330] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21: stopping tablet replica
I20260811 05:17:33.379999 5725 raft_consensus.cc:2229] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:17:33.380520 5725 raft_consensus.cc:2258] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:33.381940 5723 ts_tablet_manager.cc:1905] T d78afd01a077465887c51e78b65ab1db P ee1479c8a07544e3ae9dc09c9ff81ee6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:17:33.384989 5723 ts_tablet_manager.cc:1918] T d78afd01a077465887c51e78b65ab1db P ee1479c8a07544e3ae9dc09c9ff81ee6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:17:33.385084 5723 log.cc:1196] T d78afd01a077465887c51e78b65ab1db P ee1479c8a07544e3ae9dc09c9ff81ee6: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/d78afd01a077465887c51e78b65ab1db
I20260811 05:17:33.385437 5723 ts_tablet_manager.cc:1939] T d78afd01a077465887c51e78b65ab1db P ee1479c8a07544e3ae9dc09c9ff81ee6: Deleting consensus metadata
I20260811 05:17:33.386896 4866 catalog_manager.cc:4841] TS ee1479c8a07544e3ae9dc09c9ff81ee6 (127.7.67.131:39473): tablet d78afd01a077465887c51e78b65ab1db (table test-table [id=470ed3e3dfdf48a7a640a2f5b47a57db]) successfully deleted
I20260811 05:17:33.389899 5725 ts_tablet_manager.cc:1905] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:17:33.392921 5725 ts_tablet_manager.cc:1918] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:17:33.393019 5725 log.cc:1196] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/d78afd01a077465887c51e78b65ab1db
I20260811 05:17:33.393356 5725 ts_tablet_manager.cc:1939] T d78afd01a077465887c51e78b65ab1db P 0f79c3aafad84a6594e4245fe87c9c21: Deleting consensus metadata
I20260811 05:17:33.399504 4867 catalog_manager.cc:4841] TS 0f79c3aafad84a6594e4245fe87c9c21 (127.7.67.130:41869): tablet d78afd01a077465887c51e78b65ab1db (table test-table [id=470ed3e3dfdf48a7a640a2f5b47a57db]) successfully deleted
W20260811 05:17:33.404246 4869 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.129:40995 (error 108)
I20260811 05:17:33.404505 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 5203
I20260811 05:17:33.413407 5683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:33.411221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:40995 (local address 127.0.0.1:47056)
0811 05:17:33.413136 (+ 1915us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:33.413264 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.129:40995: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1772}
W20260811 05:17:33.415481 4869 catalog_manager.cc:4584] TS 7f28e17f10754e3482b41eeb08be63ed (127.7.67.129:40995): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d78afd01a077465887c51e78b65ab1db: Network error: Client connection negotiation failed: client connection to 127.7.67.129:40995: connect: Connection refused (error 111)
I20260811 05:17:33.422765 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 5444
I20260811 05:17:33.436426 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 4850
2026-08-11T05:17:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1073 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:33 dist-test-slave-04bb krb5kdc[5731](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:33 dist-test-slave-04bb krb5kdc[5731](info): set up 2 sockets
Aug 11 05:17:33 dist-test-slave-04bb krb5kdc[5731](info): commencing operation
krb5kdc: starting...
W20260811 05:17:34.698186 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.235s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:17:34 dist-test-slave-04bb krb5kdc[5731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:17:34.827646 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:38735
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:46843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:38735
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:34.943363 5746 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:34.943665 5746 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:34.943748 5746 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:34.947753 5746 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:34.947836 5746 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:34.947888 5746 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:34.947940 5746 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:34.947978 5746 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:34.953205 5746 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:46843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:38735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:38735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5746
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:34.954250 5746 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:34.955278 5746 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:34.963966 5761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:34.963972 5755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:34.964089 5759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:34.964365 5746 server_base.cc:1034] running on GCE node
I20260811 05:17:34.964974 5746 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:34.966142 5746 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:34.967334 5746 hybrid_clock.cc:648] HybridClock initialized: now 1786425454967313 us; error 31 us; skew 500 ppm
Aug 11 05:17:34 dist-test-slave-04bb krb5kdc[5731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425454, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:34.970335 5746 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:17:34.971279 5746 webserver.cc:460] Webserver started at http://127.7.67.190:35213/ using document root <none> and password file <none>
I20260811 05:17:34.971853 5746 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:34.971943 5746 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:34.972195 5746 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:34.974413 5746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "ad1afab93be44e5ebd0a56bbd8ec4cd7"
format_stamp: "Formatted at 2026-08-11 05:17:34 on dist-test-slave-04bb"
server_key: "bfd345ae65b56a1462736ae3eae685d1"
server_key_iv: "6ade38871ecf0d88811eead43e60623f"
server_key_version: "encryptionkey@0"
I20260811 05:17:34.974972 5746 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "ad1afab93be44e5ebd0a56bbd8ec4cd7"
format_stamp: "Formatted at 2026-08-11 05:17:34 on dist-test-slave-04bb"
server_key: "bfd345ae65b56a1462736ae3eae685d1"
server_key_iv: "6ade38871ecf0d88811eead43e60623f"
server_key_version: "encryptionkey@0"
I20260811 05:17:34.978909 5746 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 05:17:34.981632 5771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:34.982972 5746 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:17:34.983142 5746 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "ad1afab93be44e5ebd0a56bbd8ec4cd7"
format_stamp: "Formatted at 2026-08-11 05:17:34 on dist-test-slave-04bb"
server_key: "bfd345ae65b56a1462736ae3eae685d1"
server_key_iv: "6ade38871ecf0d88811eead43e60623f"
server_key_version: "encryptionkey@0"
I20260811 05:17:34.983269 5746 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:35.000751 5746 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:35.003633 5746 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:35.003840 5746 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:35.011466 5746 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:38735
I20260811 05:17:35.011480 5851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:38735 every 8 connection(s)
I20260811 05:17:35.012472 5746 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:35.014742 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 5746
I20260811 05:17:35.014900 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:17:35.015240 7438 external_mini_cluster.cc:1428] Setting key 95f96f844f9f403e485940c9c0ccaffb
I20260811 05:17:35.015578 5852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:35.023150 5852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad1afab93be44e5ebd0a56bbd8ec4cd7: Bootstrap starting.
Aug 11 05:17:35 dist-test-slave-04bb krb5kdc[5731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:17:35.026098 5852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad1afab93be44e5ebd0a56bbd8ec4cd7: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:35.027097 5852 log.cc:824] T 00000000000000000000000000000000 P ad1afab93be44e5ebd0a56bbd8ec4cd7: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:35.029444 5852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad1afab93be44e5ebd0a56bbd8ec4cd7: No bootstrap required, opened a new log
I20260811 05:17:35.031108 5856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:35.016748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42188 (local address 127.7.67.190:38735)
0811 05:17:35.017210 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:35.017230 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:35.017295 (+ 65us) server_negotiation.cc:407] Connection header received
0811 05:17:35.018081 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:35.018104 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:35.018536 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:35.018909 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:35.019623 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.021142 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.022010 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.022703 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.022868 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:35.025655 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:35.025662 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:35.025679 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:35.025711 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:35.027775 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:35.028215 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:35.028221 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:35.028227 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:35.028373 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:35.028797 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:35.028800 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:35.028802 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:35.029317 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:35.029510 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:35.029855 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 05:17:35.030160 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":139,"threads_started":1}
I20260811 05:17:35.033768 5852 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad1afab93be44e5ebd0a56bbd8ec4cd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1afab93be44e5ebd0a56bbd8ec4cd7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38735 } }
I20260811 05:17:35.033986 5852 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad1afab93be44e5ebd0a56bbd8ec4cd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:35.034029 5852 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad1afab93be44e5ebd0a56bbd8ec4cd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad1afab93be44e5ebd0a56bbd8ec4cd7, State: Initialized, Role: FOLLOWER
I20260811 05:17:35.034389 5852 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad1afab93be44e5ebd0a56bbd8ec4cd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1afab93be44e5ebd0a56bbd8ec4cd7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38735 } }
I20260811 05:17:35.034499 5852 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad1afab93be44e5ebd0a56bbd8ec4cd7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:35.034543 5852 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad1afab93be44e5ebd0a56bbd8ec4cd7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:35.034611 5852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad1afab93be44e5ebd0a56bbd8ec4cd7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:35.035615 5852 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad1afab93be44e5ebd0a56bbd8ec4cd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1afab93be44e5ebd0a56bbd8ec4cd7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38735 } }
I20260811 05:17:35.035915 5852 leader_election.cc:302] T 00000000000000000000000000000000 P ad1afab93be44e5ebd0a56bbd8ec4cd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad1afab93be44e5ebd0a56bbd8ec4cd7; no voters:
I20260811 05:17:35.036166 5852 leader_election.cc:288] T 00000000000000000000000000000000 P ad1afab93be44e5ebd0a56bbd8ec4cd7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:35.036339 5860 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad1afab93be44e5ebd0a56bbd8ec4cd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:35.036597 5860 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad1afab93be44e5ebd0a56bbd8ec4cd7 [term 1 LEADER]: Becoming Leader. State: Replica: ad1afab93be44e5ebd0a56bbd8ec4cd7, State: Running, Role: LEADER
I20260811 05:17:35.036907 5860 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad1afab93be44e5ebd0a56bbd8ec4cd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1afab93be44e5ebd0a56bbd8ec4cd7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38735 } }
I20260811 05:17:35.037438 5852 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad1afab93be44e5ebd0a56bbd8ec4cd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:35.038648 5864 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1afab93be44e5ebd0a56bbd8ec4cd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad1afab93be44e5ebd0a56bbd8ec4cd7. Latest consensus state: current_term: 1 leader_uuid: "ad1afab93be44e5ebd0a56bbd8ec4cd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1afab93be44e5ebd0a56bbd8ec4cd7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38735 } } }
I20260811 05:17:35.038661 5863 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad1afab93be44e5ebd0a56bbd8ec4cd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad1afab93be44e5ebd0a56bbd8ec4cd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad1afab93be44e5ebd0a56bbd8ec4cd7" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 38735 } } }
I20260811 05:17:35.038800 5864 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1afab93be44e5ebd0a56bbd8ec4cd7 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:35.038800 5863 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad1afab93be44e5ebd0a56bbd8ec4cd7 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:35.039160 5876 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:35.042498 5876 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:35.047858 5876 catalog_manager.cc:1347] Generated new cluster ID: 22d94abadf354861a9d31d76108e0581
I20260811 05:17:35.047937 5876 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:35.067272 5876 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:35.068176 5876 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:35.074312 5876 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad1afab93be44e5ebd0a56bbd8ec4cd7: Generated new TSK 0
I20260811 05:17:35.074970 5876 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.7.67.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
Entry for principal kudu/127.7.67.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab.
I20260811 05:17:35.111799 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:38735
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:46843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:35.261289 5898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:35.261539 5898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:35.261601 5898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:35.265667 5898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:35.265722 5898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:35.265807 5898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:17:35.272403 5898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:46843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.129.keytab
--principal=kudu/127.7.67.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:38735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:35.273567 5898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:35.274631 5898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:35.284554 5907 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:35.284644 5906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:35.284648 5898 server_base.cc:1034] running on GCE node
W20260811 05:17:35.284855 5910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:35.285243 5898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:35.285984 5898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:35.287176 5898 hybrid_clock.cc:648] HybridClock initialized: now 1786425455287144 us; error 48 us; skew 500 ppm
Aug 11 05:17:35 dist-test-slave-04bb krb5kdc[5731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425455, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:35.290271 5898 init.cc:377] Logged in from keytab as kudu/127.7.67.129@KRBTEST.COM (short username kudu)
I20260811 05:17:35.291224 5898 webserver.cc:460] Webserver started at http://127.7.67.129:37773/ using document root <none> and password file <none>
I20260811 05:17:35.291776 5898 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:35.291838 5898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:35.292016 5898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:35.294085 5898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "5e854a099d2548eb8a4dcbb2e4a8ee36"
format_stamp: "Formatted at 2026-08-11 05:17:35 on dist-test-slave-04bb"
server_key: "b2748350af4f8c1714d95d27c5af3f48"
server_key_iv: "aff834fdef29af37c35161470033560a"
server_key_version: "encryptionkey@0"
I20260811 05:17:35.294602 5898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "5e854a099d2548eb8a4dcbb2e4a8ee36"
format_stamp: "Formatted at 2026-08-11 05:17:35 on dist-test-slave-04bb"
server_key: "b2748350af4f8c1714d95d27c5af3f48"
server_key_iv: "aff834fdef29af37c35161470033560a"
server_key_version: "encryptionkey@0"
I20260811 05:17:35.298331 5898 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 05:17:35.300849 5920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:35.302181 5898 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:17:35.302310 5898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "5e854a099d2548eb8a4dcbb2e4a8ee36"
format_stamp: "Formatted at 2026-08-11 05:17:35 on dist-test-slave-04bb"
server_key: "b2748350af4f8c1714d95d27c5af3f48"
server_key_iv: "aff834fdef29af37c35161470033560a"
server_key_version: "encryptionkey@0"
I20260811 05:17:35.302397 5898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:35.326646 5898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:35.329995 5898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:35.330154 5898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:35.330706 5898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:35.331489 5898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:35.331537 5898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:35.331575 5898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:35.331591 5898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:35.341588 5898 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:35571
I20260811 05:17:35.341702 6109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:35571 every 8 connection(s)
I20260811 05:17:35.342584 5898 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:17:35.350106 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 5898
I20260811 05:17:35.350324 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:17:35.350677 7438 external_mini_cluster.cc:1428] Setting key 985ea97a8565a63d3ef3770def851562
Aug 11 05:17:35 dist-test-slave-04bb krb5kdc[5731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425455, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.129@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:17:35.357388 5856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:35.344512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:41577 (local address 127.7.67.190:38735)
0811 05:17:35.344689 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:35.344693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:35.345554 (+ 861us) server_negotiation.cc:407] Connection header received
0811 05:17:35.346940 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:35.346943 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:35.346989 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:35.347068 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:35.348351 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.348918 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.350148 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.350530 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.350643 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:35.353384 (+ 2741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:35.353392 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:35.353398 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:35.353431 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:35.355073 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:35.355719 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:35.355722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:35.355723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:35.355800 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:35.356202 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:35.356205 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:35.356206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:35.356388 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:35.356465 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:35.357153 (+ 688us) server_negotiation.cc:299] Negotiation successful
0811 05:17:35.357268 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 05:17:35.358677 6112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:35.344926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38735 (local address 127.7.67.129:41577)
0811 05:17:35.345392 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:35.345439 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:35.346716 (+ 1277us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:35.347292 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:35.347299 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:35.347668 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:35.348187 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:35.348202 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.349035 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.349038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:35.350017 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:35.350028 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.350690 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.350693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:35.350886 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:35.351606 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:35.351622 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:35.353210 (+ 1588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:35.355196 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:35.355203 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:35.355220 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:35.355617 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:35.355894 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:35.355900 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:35.355905 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:35.356098 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:35.356496 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:35.356506 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:35.356875 (+ 369us) client_negotiation.cc:769] Sending connection context
0811 05:17:35.357094 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 05:17:35.357487 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":108,"threads_started":1}
I20260811 05:17:35.359984 6110 heartbeater.cc:347] Connected to a master server at 127.7.67.190:38735
I20260811 05:17:35.360262 6110 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:35.360946 6110 heartbeater.cc:511] Master 127.7.67.190:38735 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.7.67.130@KRBTEST.COM; defaulting to no policy
I20260811 05:17:35.362668 5798 ts_manager.cc:194] Registered new tserver with Master: 5e854a099d2548eb8a4dcbb2e4a8ee36 (127.7.67.129:35571)
I20260811 05:17:35.363878 5798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.129@KRBTEST.COM'} at 127.7.67.129:41577
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
Entry for principal kudu/127.7.67.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab.
I20260811 05:17:35.383371 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:38735
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:46843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:35.498415 6117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:35.498660 6117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:35.498703 6117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:35.502665 6117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:35.502718 6117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:35.502779 6117 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:17:35.508029 6117 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:46843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.130.keytab
--principal=kudu/127.7.67.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:38735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6117
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:35.509039 6117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:35.510017 6117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:35.519754 6128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:35.519865 6124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:35.519754 6123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:35.520133 6117 server_base.cc:1034] running on GCE node
I20260811 05:17:35.520651 6117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:35.521332 6117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:35.522539 6117 hybrid_clock.cc:648] HybridClock initialized: now 1786425455522482 us; error 64 us; skew 500 ppm
Aug 11 05:17:35 dist-test-slave-04bb krb5kdc[5731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425455, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:35.525514 6117 init.cc:377] Logged in from keytab as kudu/127.7.67.130@KRBTEST.COM (short username kudu)
I20260811 05:17:35.526470 6117 webserver.cc:460] Webserver started at http://127.7.67.130:38123/ using document root <none> and password file <none>
I20260811 05:17:35.527084 6117 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:35.527182 6117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:35.527434 6117 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:35.529755 6117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "c04c5e7f9d4f480d94f0ef50ceb17171"
format_stamp: "Formatted at 2026-08-11 05:17:35 on dist-test-slave-04bb"
server_key: "d36fec44fb510f0b18eb8d63850fa10f"
server_key_iv: "a3f89eb51321270f208896a448de87d8"
server_key_version: "encryptionkey@0"
I20260811 05:17:35.530442 6117 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "c04c5e7f9d4f480d94f0ef50ceb17171"
format_stamp: "Formatted at 2026-08-11 05:17:35 on dist-test-slave-04bb"
server_key: "d36fec44fb510f0b18eb8d63850fa10f"
server_key_iv: "a3f89eb51321270f208896a448de87d8"
server_key_version: "encryptionkey@0"
I20260811 05:17:35.534579 6117 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 05:17:35.537267 6137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:35.538637 6117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:17:35.538767 6117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "c04c5e7f9d4f480d94f0ef50ceb17171"
format_stamp: "Formatted at 2026-08-11 05:17:35 on dist-test-slave-04bb"
server_key: "d36fec44fb510f0b18eb8d63850fa10f"
server_key_iv: "a3f89eb51321270f208896a448de87d8"
server_key_version: "encryptionkey@0"
I20260811 05:17:35.538906 6117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:35.551689 6117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:35.555058 6117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:35.555266 6117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:35.555869 6117 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:35.556797 6117 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:35.556845 6117 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:35.556902 6117 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:35.556936 6117 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:35.565913 6117 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:33367
I20260811 05:17:35.565973 6316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:33367 every 8 connection(s)
I20260811 05:17:35.566900 6117 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:17:35.570780 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 6117
I20260811 05:17:35.570909 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:17:35.571261 7438 external_mini_cluster.cc:1428] Setting key f945c66ed17b252132c1a749af258b25
Aug 11 05:17:35 dist-test-slave-04bb krb5kdc[5731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425455, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.130@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.131@KRBTEST.COM; defaulting to no policy
I20260811 05:17:35.583302 5856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:35.568826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:46351 (local address 127.7.67.190:38735)
0811 05:17:35.568966 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:35.568974 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:35.569853 (+ 879us) server_negotiation.cc:407] Connection header received
0811 05:17:35.571524 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:35.571527 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:35.571577 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:35.571678 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:35.573207 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.573914 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.575578 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.576031 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.576180 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:35.579264 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:35.579268 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:35.579270 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:35.579296 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:35.580965 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:35.581548 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:35.581553 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:35.581555 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:35.581656 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:35.582035 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:35.582039 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:35.582041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:35.582253 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:35.582368 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:35.582975 (+ 607us) server_negotiation.cc:299] Negotiation successful
0811 05:17:35.583115 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 05:17:35.584235 6323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:35.569210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38735 (local address 127.7.67.130:46351)
0811 05:17:35.569705 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:35.569749 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:35.571285 (+ 1536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:35.571831 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:35.571838 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:35.572350 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:35.573018 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:35.573039 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.574086 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.574093 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:35.575438 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:35.575449 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.576203 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.576207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:35.576363 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:35.577073 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:35.577099 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:35.579041 (+ 1942us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:35.581108 (+ 2067us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:35.581114 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:35.581141 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:35.581430 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:35.581761 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:35.581766 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:35.581768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:35.581927 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:35.582368 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:35.582378 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:35.582699 (+ 321us) client_negotiation.cc:769] Sending connection context
0811 05:17:35.582906 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 05:17:35.583289 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":124,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.131@KRBTEST.COM" created.
I20260811 05:17:35.585695 6319 heartbeater.cc:347] Connected to a master server at 127.7.67.190:38735
I20260811 05:17:35.585983 6319 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:35.586580 6319 heartbeater.cc:511] Master 127.7.67.190:38735 requested a full tablet report, sending...
I20260811 05:17:35.587776 5798 ts_manager.cc:194] Registered new tserver with Master: c04c5e7f9d4f480d94f0ef50ceb17171 (127.7.67.130:33367)
I20260811 05:17:35.588485 5798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.130@KRBTEST.COM'} at 127.7.67.130:46351
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
Entry for principal kudu/127.7.67.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab.
I20260811 05:17:35.603720 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:38735
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:46843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:35.718161 6327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:35.718398 6327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:35.718439 6327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:35.722419 6327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:35.722469 6327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:35.722525 6327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:17:35.727566 6327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:46843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.131.keytab
--principal=kudu/127.7.67.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:38735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:35.728567 6327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:35.729561 6327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:35.739944 6340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:35.739969 6336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:35.739967 6344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:35.740451 6327 server_base.cc:1034] running on GCE node
I20260811 05:17:35.740911 6327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:35.741627 6327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:35.742817 6327 hybrid_clock.cc:648] HybridClock initialized: now 1786425455742798 us; error 37 us; skew 500 ppm
Aug 11 05:17:35 dist-test-slave-04bb krb5kdc[5731](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425455, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:35.746095 6327 init.cc:377] Logged in from keytab as kudu/127.7.67.131@KRBTEST.COM (short username kudu)
I20260811 05:17:35.746990 6327 webserver.cc:460] Webserver started at http://127.7.67.131:41447/ using document root <none> and password file <none>
I20260811 05:17:35.747587 6327 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:35.747653 6327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:35.747833 6327 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:35.750067 6327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "c0582d9d943c422fb5efb936515eb1a9"
format_stamp: "Formatted at 2026-08-11 05:17:35 on dist-test-slave-04bb"
server_key: "52a4ce6f9e70660604296ae41fa61fed"
server_key_iv: "fc67e5125e0c2045ec043779463708c5"
server_key_version: "encryptionkey@0"
I20260811 05:17:35.750651 6327 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "c0582d9d943c422fb5efb936515eb1a9"
format_stamp: "Formatted at 2026-08-11 05:17:35 on dist-test-slave-04bb"
server_key: "52a4ce6f9e70660604296ae41fa61fed"
server_key_iv: "fc67e5125e0c2045ec043779463708c5"
server_key_version: "encryptionkey@0"
I20260811 05:17:35.754638 6327 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 05:17:35.757171 6355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:35.758481 6327 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:17:35.758637 6327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "c0582d9d943c422fb5efb936515eb1a9"
format_stamp: "Formatted at 2026-08-11 05:17:35 on dist-test-slave-04bb"
server_key: "52a4ce6f9e70660604296ae41fa61fed"
server_key_iv: "fc67e5125e0c2045ec043779463708c5"
server_key_version: "encryptionkey@0"
I20260811 05:17:35.758747 6327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:35.778192 6327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:35.781952 6327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:35.782176 6327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:35.782838 6327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:35.783746 6327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:35.783823 6327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:35.783898 6327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:35.783947 6327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:35.794704 6327 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:34417
I20260811 05:17:35.794739 6536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:34417 every 8 connection(s)
I20260811 05:17:35.795683 6327 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:17:35.801318 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 6327
I20260811 05:17:35.801492 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:17:35.801877 7438 external_mini_cluster.cc:1428] Setting key 788ee445b45a4c2c2e0340ce358c35c7
Aug 11 05:17:35 dist-test-slave-04bb krb5kdc[5731](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425455, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.131@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:17:35.810917 5856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:35.797764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:47601 (local address 127.7.67.190:38735)
0811 05:17:35.797929 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:35.797934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:35.798927 (+ 993us) server_negotiation.cc:407] Connection header received
0811 05:17:35.800402 (+ 1475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:35.800406 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:35.800466 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:35.800595 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:35.802400 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.803005 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.804316 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.804653 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.804781 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:35.807502 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:35.807505 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:35.807507 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:35.807531 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:35.808890 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:35.809539 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:35.809542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:35.809543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:35.809624 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:35.809949 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:35.809952 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:35.809953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:35.810130 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:35.810202 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:35.810694 (+ 492us) server_negotiation.cc:299] Negotiation successful
0811 05:17:35.810797 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 05:17:35.811712 6539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:35.798239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:38735 (local address 127.7.67.131:47601)
0811 05:17:35.798757 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:35.798806 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:35.800170 (+ 1364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:35.800763 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:35.800771 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:35.801369 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:35.802150 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:35.802175 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.803167 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.803172 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:35.804163 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:35.804174 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.804838 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.804844 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:35.805041 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:35.805639 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:17:35.805658 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:35.807335 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:35.808988 (+ 1653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:35.808993 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:35.809005 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:35.809413 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:35.809717 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:17:35.809720 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:17:35.809722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:17:35.809868 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:17:35.810250 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:35.810256 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:35.810481 (+ 225us) client_negotiation.cc:769] Sending connection context
0811 05:17:35.810634 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 05:17:35.810886 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":113,"threads_started":1}
I20260811 05:17:35.812855 6537 heartbeater.cc:347] Connected to a master server at 127.7.67.190:38735
I20260811 05:17:35.813128 6537 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:35.813690 6537 heartbeater.cc:511] Master 127.7.67.190:38735 requested a full tablet report, sending...
I20260811 05:17:35.814827 5798 ts_manager.cc:194] Registered new tserver with Master: c0582d9d943c422fb5efb936515eb1a9 (127.7.67.131:34417)
I20260811 05:17:35.815569 5798 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.7.67.131@KRBTEST.COM'} at 127.7.67.131:47601
I20260811 05:17:35.815941 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:17:35.825017 5856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:35.818052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45902 (local address 127.7.67.190:38735)
0811 05:17:35.818162 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:35.818171 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:35.818332 (+ 161us) server_negotiation.cc:407] Connection header received
0811 05:17:35.818425 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:35.818427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:35.818463 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:35.818526 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:35.819154 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.819652 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.820494 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.820772 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.820855 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:35.822078 (+ 1223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:35.822082 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:35.822083 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:35.822105 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:35.823502 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:35.823909 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:35.823912 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:35.823913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:35.823995 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:35.824243 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:35.824246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:35.824247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:35.824461 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:35.824639 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:35.824810 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 05:17:35.824912 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 05:17:35.827651 5798 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45902:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:17:35.830753 5798 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:17:35.845858 6554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:35.839949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54974 (local address 127.7.67.130:33367)
0811 05:17:35.840388 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:35.840400 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:35.840830 (+ 430us) server_negotiation.cc:407] Connection header received
0811 05:17:35.841149 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:35.841156 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:35.841398 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:35.841560 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:35.841874 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.842986 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.844252 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.845103 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.845225 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:35.845384 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:35.845597 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 05:17:35.845688 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":120,"threads_started":1}
I20260811 05:17:35.845850 6553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:35.839797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:33367 (local address 127.0.0.1:54974)
0811 05:17:35.840672 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:35.840733 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:35.840949 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:35.841555 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:35.841561 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:35.841585 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:35.841747 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:35.841758 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.843112 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.843115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:35.844114 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:35.844124 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.845220 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.845224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:35.845391 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:35.845438 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 05:17:35.845564 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 05:17:35.845689 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"mutex_wait_us":31,"thread_start_us":95,"threads_started":1}
I20260811 05:17:35.846810 6555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:35.840149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:35571 (local address 127.0.0.1:40612)
0811 05:17:35.841170 (+ 1021us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:35.841183 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:35.841288 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:35.841997 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:35.842000 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:35.842027 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:35.842269 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:35.842277 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.844273 (+ 1996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.844276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:35.845136 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:35.845145 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.846418 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.846420 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:35.846493 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:35.846510 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 05:17:35.846597 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 05:17:35.846668 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":960,"mutex_wait_us":84,"spinlock_wait_cycles":2304,"thread_start_us":105,"threads_started":1}
I20260811 05:17:35.846884 6556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:35.840733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40612 (local address 127.7.67.129:35571)
0811 05:17:35.841167 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:35.841182 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:35.841264 (+ 82us) server_negotiation.cc:407] Connection header received
0811 05:17:35.841424 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:35.841428 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:35.841657 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:35.841811 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:35.842806 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.844115 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.845291 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.846088 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.846191 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:35.846364 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:35.846634 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 05:17:35.846735 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":93,"threads_started":1}
I20260811 05:17:35.847689 6557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:35.840772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:34417 (local address 127.0.0.1:57280)
0811 05:17:35.841187 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:35.841201 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:35.841302 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:35.842036 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:35.842039 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:35.842051 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:35.842283 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:35.842290 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.845009 (+ 2719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.845013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:35.846199 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:35.846210 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.847292 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.847296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:35.847392 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:35.847415 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 05:17:35.847478 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 05:17:35.847540 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":52,"spinlock_wait_cycles":6912,"thread_start_us":104,"threads_started":1}
I20260811 05:17:35.848399 6558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:35.840783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57280 (local address 127.7.67.131:34417)
0811 05:17:35.841550 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:35.841561 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:35.841582 (+ 21us) server_negotiation.cc:407] Connection header received
0811 05:17:35.841655 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:35.841660 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:35.841871 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:35.842071 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:35.842379 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.844866 (+ 2487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.846326 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.847164 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.847278 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:35.848096 (+ 818us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:35.848157 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 05:17:35.848253 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":667,"thread_start_us":108,"threads_started":1}
I20260811 05:17:35.849964 6211 tablet_service.cc:1467] Processing CreateTablet for tablet 0fcdfbd3a58b49c0b937d28543d519e2 (DEFAULT_TABLE table=test-table [id=92c6d54960c243019f67d3aebb9afe7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:35.850010 5996 tablet_service.cc:1467] Processing CreateTablet for tablet 0fcdfbd3a58b49c0b937d28543d519e2 (DEFAULT_TABLE table=test-table [id=92c6d54960c243019f67d3aebb9afe7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:35.851078 6211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fcdfbd3a58b49c0b937d28543d519e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:35.851105 5996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fcdfbd3a58b49c0b937d28543d519e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:35.851065 6428 tablet_service.cc:1467] Processing CreateTablet for tablet 0fcdfbd3a58b49c0b937d28543d519e2 (DEFAULT_TABLE table=test-table [id=92c6d54960c243019f67d3aebb9afe7a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:35.852015 6428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fcdfbd3a58b49c0b937d28543d519e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:35.857566 6560 tablet_bootstrap.cc:492] T 0fcdfbd3a58b49c0b937d28543d519e2 P c04c5e7f9d4f480d94f0ef50ceb17171: Bootstrap starting.
I20260811 05:17:35.859324 6562 tablet_bootstrap.cc:492] T 0fcdfbd3a58b49c0b937d28543d519e2 P c0582d9d943c422fb5efb936515eb1a9: Bootstrap starting.
I20260811 05:17:35.859975 6560 tablet_bootstrap.cc:654] T 0fcdfbd3a58b49c0b937d28543d519e2 P c04c5e7f9d4f480d94f0ef50ceb17171: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:35.860874 6561 tablet_bootstrap.cc:492] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36: Bootstrap starting.
I20260811 05:17:35.861302 6562 tablet_bootstrap.cc:654] T 0fcdfbd3a58b49c0b937d28543d519e2 P c0582d9d943c422fb5efb936515eb1a9: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:35.861315 6560 log.cc:824] T 0fcdfbd3a58b49c0b937d28543d519e2 P c04c5e7f9d4f480d94f0ef50ceb17171: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:35.862607 6562 log.cc:824] T 0fcdfbd3a58b49c0b937d28543d519e2 P c0582d9d943c422fb5efb936515eb1a9: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:35.863152 6561 tablet_bootstrap.cc:654] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:35.863529 6560 tablet_bootstrap.cc:492] T 0fcdfbd3a58b49c0b937d28543d519e2 P c04c5e7f9d4f480d94f0ef50ceb17171: No bootstrap required, opened a new log
I20260811 05:17:35.863742 6560 ts_tablet_manager.cc:1397] T 0fcdfbd3a58b49c0b937d28543d519e2 P c04c5e7f9d4f480d94f0ef50ceb17171: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 05:17:35.864041 6561 log.cc:824] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:35.864764 6562 tablet_bootstrap.cc:492] T 0fcdfbd3a58b49c0b937d28543d519e2 P c0582d9d943c422fb5efb936515eb1a9: No bootstrap required, opened a new log
I20260811 05:17:35.864970 6562 ts_tablet_manager.cc:1397] T 0fcdfbd3a58b49c0b937d28543d519e2 P c0582d9d943c422fb5efb936515eb1a9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260811 05:17:35.865989 6561 tablet_bootstrap.cc:492] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36: No bootstrap required, opened a new log
I20260811 05:17:35.866175 6561 ts_tablet_manager.cc:1397] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:17:35.867107 6560 raft_consensus.cc:348] T 0fcdfbd3a58b49c0b937d28543d519e2 P c04c5e7f9d4f480d94f0ef50ceb17171 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c04c5e7f9d4f480d94f0ef50ceb17171" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 33367 } } peers { permanent_uuid: "c0582d9d943c422fb5efb936515eb1a9" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 34417 } } peers { permanent_uuid: "5e854a099d2548eb8a4dcbb2e4a8ee36" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 35571 } }
I20260811 05:17:35.867359 6560 raft_consensus.cc:374] T 0fcdfbd3a58b49c0b937d28543d519e2 P c04c5e7f9d4f480d94f0ef50ceb17171 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:35.867414 6560 raft_consensus.cc:729] T 0fcdfbd3a58b49c0b937d28543d519e2 P c04c5e7f9d4f480d94f0ef50ceb17171 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c04c5e7f9d4f480d94f0ef50ceb17171, State: Initialized, Role: FOLLOWER
I20260811 05:17:35.867945 6560 consensus_queue.cc:260] T 0fcdfbd3a58b49c0b937d28543d519e2 P c04c5e7f9d4f480d94f0ef50ceb17171 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c04c5e7f9d4f480d94f0ef50ceb17171" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 33367 } } peers { permanent_uuid: "c0582d9d943c422fb5efb936515eb1a9" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 34417 } } peers { permanent_uuid: "5e854a099d2548eb8a4dcbb2e4a8ee36" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 35571 } }
I20260811 05:17:35.868983 6560 ts_tablet_manager.cc:1428] T 0fcdfbd3a58b49c0b937d28543d519e2 P c04c5e7f9d4f480d94f0ef50ceb17171: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260811 05:17:35.868868 6562 raft_consensus.cc:348] T 0fcdfbd3a58b49c0b937d28543d519e2 P c0582d9d943c422fb5efb936515eb1a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c04c5e7f9d4f480d94f0ef50ceb17171" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 33367 } } peers { permanent_uuid: "c0582d9d943c422fb5efb936515eb1a9" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 34417 } } peers { permanent_uuid: "5e854a099d2548eb8a4dcbb2e4a8ee36" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 35571 } }
I20260811 05:17:35.869129 6562 raft_consensus.cc:374] T 0fcdfbd3a58b49c0b937d28543d519e2 P c0582d9d943c422fb5efb936515eb1a9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:35.869200 6562 raft_consensus.cc:729] T 0fcdfbd3a58b49c0b937d28543d519e2 P c0582d9d943c422fb5efb936515eb1a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0582d9d943c422fb5efb936515eb1a9, State: Initialized, Role: FOLLOWER
I20260811 05:17:35.869397 6319 heartbeater.cc:503] Master 127.7.67.190:38735 was elected leader, sending a full tablet report...
I20260811 05:17:35.869625 6562 consensus_queue.cc:260] T 0fcdfbd3a58b49c0b937d28543d519e2 P c0582d9d943c422fb5efb936515eb1a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c04c5e7f9d4f480d94f0ef50ceb17171" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 33367 } } peers { permanent_uuid: "c0582d9d943c422fb5efb936515eb1a9" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 34417 } } peers { permanent_uuid: "5e854a099d2548eb8a4dcbb2e4a8ee36" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 35571 } }
I20260811 05:17:35.870294 6537 heartbeater.cc:503] Master 127.7.67.190:38735 was elected leader, sending a full tablet report...
I20260811 05:17:35.870569 6562 ts_tablet_manager.cc:1428] T 0fcdfbd3a58b49c0b937d28543d519e2 P c0582d9d943c422fb5efb936515eb1a9: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260811 05:17:35.870556 6561 raft_consensus.cc:348] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c04c5e7f9d4f480d94f0ef50ceb17171" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 33367 } } peers { permanent_uuid: "c0582d9d943c422fb5efb936515eb1a9" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 34417 } } peers { permanent_uuid: "5e854a099d2548eb8a4dcbb2e4a8ee36" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 35571 } }
I20260811 05:17:35.870792 6561 raft_consensus.cc:374] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:35.870854 6561 raft_consensus.cc:729] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5e854a099d2548eb8a4dcbb2e4a8ee36, State: Initialized, Role: FOLLOWER
I20260811 05:17:35.871243 6561 consensus_queue.cc:260] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c04c5e7f9d4f480d94f0ef50ceb17171" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 33367 } } peers { permanent_uuid: "c0582d9d943c422fb5efb936515eb1a9" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 34417 } } peers { permanent_uuid: "5e854a099d2548eb8a4dcbb2e4a8ee36" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 35571 } }
I20260811 05:17:35.871922 6110 heartbeater.cc:503] Master 127.7.67.190:38735 was elected leader, sending a full tablet report...
I20260811 05:17:35.872148 6561 ts_tablet_manager.cc:1428] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36: Time spent starting tablet: real 0.006s user 0.000s sys 0.007s
I20260811 05:17:35.953837 6577 raft_consensus.cc:482] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:17:35.954103 6577 raft_consensus.cc:504] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c04c5e7f9d4f480d94f0ef50ceb17171" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 33367 } } peers { permanent_uuid: "c0582d9d943c422fb5efb936515eb1a9" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 34417 } } peers { permanent_uuid: "5e854a099d2548eb8a4dcbb2e4a8ee36" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 35571 } }
I20260811 05:17:35.955333 6577 leader_election.cc:288] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c04c5e7f9d4f480d94f0ef50ceb17171 (127.7.67.130:33367), c0582d9d943c422fb5efb936515eb1a9 (127.7.67.131:34417)
I20260811 05:17:35.959542 6554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:35.955690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:59303 (local address 127.7.67.130:33367)
0811 05:17:35.955849 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:35.955854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:35.956063 (+ 209us) server_negotiation.cc:407] Connection header received
0811 05:17:35.956456 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:35.956459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:35.956502 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:35.956617 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:35.956915 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.957432 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.958521 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.958924 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.959054 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:35.959120 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:35.959309 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 05:17:35.959381 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:17:35.959710 6579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:35.956120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:34417 (local address 127.7.67.129:54935)
0811 05:17:35.956481 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:35.956496 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:35.956623 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:35.956884 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:35.956887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:35.956902 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:35.957023 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:35.957029 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.957726 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.957730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:35.958544 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:35.958553 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.959242 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.959247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:35.959352 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:35.959373 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 05:17:35.959480 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 05:17:35.959544 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":27,"thread_start_us":124,"threads_started":1}
I20260811 05:17:35.959723 6558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:35.956327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:54935 (local address 127.7.67.131:34417)
0811 05:17:35.956484 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:35.956491 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:35.956576 (+ 85us) server_negotiation.cc:407] Connection header received
0811 05:17:35.956736 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:35.956744 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:35.956786 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:35.956862 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:35.957132 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.957616 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.958670 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.959127 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.959268 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:35.959320 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:35.959527 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 05:17:35.959606 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 05:17:35.959934 6578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:35.955610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:33367 (local address 127.7.67.129:59303)
0811 05:17:35.955953 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:35.955967 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:35.956109 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:35.956634 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:35.956637 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:35.956664 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:35.956804 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:35.956810 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.957541 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.957546 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:35.958395 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:35.958404 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.959054 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.959058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:35.959153 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:35.959178 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 05:17:35.959690 (+ 512us) client_negotiation.cc:240] Negotiation successful
0811 05:17:35.959771 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":5,"thread_start_us":155,"threads_started":1}
I20260811 05:17:35.960507 6241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fcdfbd3a58b49c0b937d28543d519e2" candidate_uuid: "5e854a099d2548eb8a4dcbb2e4a8ee36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c04c5e7f9d4f480d94f0ef50ceb17171" is_pre_election: true
I20260811 05:17:35.960574 6455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fcdfbd3a58b49c0b937d28543d519e2" candidate_uuid: "5e854a099d2548eb8a4dcbb2e4a8ee36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0582d9d943c422fb5efb936515eb1a9" is_pre_election: true
I20260811 05:17:35.960866 6241 raft_consensus.cc:2454] T 0fcdfbd3a58b49c0b937d28543d519e2 P c04c5e7f9d4f480d94f0ef50ceb17171 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e854a099d2548eb8a4dcbb2e4a8ee36 in term 0.
I20260811 05:17:35.960886 6455 raft_consensus.cc:2454] T 0fcdfbd3a58b49c0b937d28543d519e2 P c0582d9d943c422fb5efb936515eb1a9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5e854a099d2548eb8a4dcbb2e4a8ee36 in term 0.
I20260811 05:17:35.961364 5923 leader_election.cc:302] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e854a099d2548eb8a4dcbb2e4a8ee36, c04c5e7f9d4f480d94f0ef50ceb17171; no voters:
I20260811 05:17:35.961647 6577 raft_consensus.cc:2781] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:17:35.961725 6577 raft_consensus.cc:482] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:17:35.961782 6577 raft_consensus.cc:3037] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:35.962944 6577 raft_consensus.cc:504] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c04c5e7f9d4f480d94f0ef50ceb17171" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 33367 } } peers { permanent_uuid: "c0582d9d943c422fb5efb936515eb1a9" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 34417 } } peers { permanent_uuid: "5e854a099d2548eb8a4dcbb2e4a8ee36" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 35571 } }
I20260811 05:17:35.963325 6577 leader_election.cc:288] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36 [CANDIDATE]: Term 1 election: Requested vote from peers c04c5e7f9d4f480d94f0ef50ceb17171 (127.7.67.130:33367), c0582d9d943c422fb5efb936515eb1a9 (127.7.67.131:34417)
I20260811 05:17:35.963766 6455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fcdfbd3a58b49c0b937d28543d519e2" candidate_uuid: "5e854a099d2548eb8a4dcbb2e4a8ee36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0582d9d943c422fb5efb936515eb1a9"
I20260811 05:17:35.963801 6241 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fcdfbd3a58b49c0b937d28543d519e2" candidate_uuid: "5e854a099d2548eb8a4dcbb2e4a8ee36" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c04c5e7f9d4f480d94f0ef50ceb17171"
I20260811 05:17:35.963900 6455 raft_consensus.cc:3037] T 0fcdfbd3a58b49c0b937d28543d519e2 P c0582d9d943c422fb5efb936515eb1a9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:35.963927 6241 raft_consensus.cc:3037] T 0fcdfbd3a58b49c0b937d28543d519e2 P c04c5e7f9d4f480d94f0ef50ceb17171 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:35.965021 6455 raft_consensus.cc:2454] T 0fcdfbd3a58b49c0b937d28543d519e2 P c0582d9d943c422fb5efb936515eb1a9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e854a099d2548eb8a4dcbb2e4a8ee36 in term 1.
I20260811 05:17:35.965209 6241 raft_consensus.cc:2454] T 0fcdfbd3a58b49c0b937d28543d519e2 P c04c5e7f9d4f480d94f0ef50ceb17171 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5e854a099d2548eb8a4dcbb2e4a8ee36 in term 1.
I20260811 05:17:35.965387 5923 leader_election.cc:302] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5e854a099d2548eb8a4dcbb2e4a8ee36, c0582d9d943c422fb5efb936515eb1a9; no voters:
I20260811 05:17:35.965564 6577 raft_consensus.cc:2781] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:35.965791 6577 raft_consensus.cc:686] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36 [term 1 LEADER]: Becoming Leader. State: Replica: 5e854a099d2548eb8a4dcbb2e4a8ee36, State: Running, Role: LEADER
I20260811 05:17:35.966138 6577 consensus_queue.cc:237] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c04c5e7f9d4f480d94f0ef50ceb17171" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 33367 } } peers { permanent_uuid: "c0582d9d943c422fb5efb936515eb1a9" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 34417 } } peers { permanent_uuid: "5e854a099d2548eb8a4dcbb2e4a8ee36" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 35571 } }
I20260811 05:17:35.969287 5798 catalog_manager.cc:5495] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5e854a099d2548eb8a4dcbb2e4a8ee36 (127.7.67.129). New cstate: current_term: 1 leader_uuid: "5e854a099d2548eb8a4dcbb2e4a8ee36" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c04c5e7f9d4f480d94f0ef50ceb17171" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 33367 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0582d9d943c422fb5efb936515eb1a9" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 34417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5e854a099d2548eb8a4dcbb2e4a8ee36" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 35571 } health_report { overall_health: HEALTHY } } }
I20260811 05:17:35.998373 6556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:35.994529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40626 (local address 127.7.67.129:35571)
0811 05:17:35.994679 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:35.994683 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:35.994698 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:17:35.994863 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:35.994866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:35.994922 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:35.995013 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:17:35.995311 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.995804 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.996827 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:35.997041 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:35.997122 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:35.997459 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:17:35.997894 (+ 435us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:17:35.997988 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:35.998088 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 05:17:35.998141 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 05:17:36.004778 6241 raft_consensus.cc:1261] T 0fcdfbd3a58b49c0b937d28543d519e2 P c04c5e7f9d4f480d94f0ef50ceb17171 [term 1 FOLLOWER]: Refusing update from remote peer 5e854a099d2548eb8a4dcbb2e4a8ee36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:36.004806 6455 raft_consensus.cc:1261] T 0fcdfbd3a58b49c0b937d28543d519e2 P c0582d9d943c422fb5efb936515eb1a9 [term 1 FOLLOWER]: Refusing update from remote peer 5e854a099d2548eb8a4dcbb2e4a8ee36: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:36.005555 6577 consensus_queue.cc:1035] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c04c5e7f9d4f480d94f0ef50ceb17171" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 33367 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:36.005740 6582 consensus_queue.cc:1035] T 0fcdfbd3a58b49c0b937d28543d519e2 P 5e854a099d2548eb8a4dcbb2e4a8ee36 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0582d9d943c422fb5efb936515eb1a9" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 34417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:36.013957 6590 mvcc.cc:205] Tried to move back new op lower bound from 7317198667788034048 to 7317198667639975936. Current Snapshot: MvccSnapshot[applied={T|T < 7317198667788034048}]
I20260811 05:17:36.015599 6592 mvcc.cc:205] Tried to move back new op lower bound from 7317198667788034048 to 7317198667639975936. Current Snapshot: MvccSnapshot[applied={T|T < 7317198667788034048}]
I20260811 05:17:36.020675 6591 mvcc.cc:205] Tried to move back new op lower bound from 7317198667788034048 to 7317198667639975936. Current Snapshot: MvccSnapshot[applied={T|T < 7317198667788034048 or (T in {7317198667788034048})}]
I20260811 05:17:36.029111 5796 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:17:36.029270 5796 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45902:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:17:36.031840 5796 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad1afab93be44e5ebd0a56bbd8ec4cd7: Sending DeleteTablet for 3 replicas of tablet 0fcdfbd3a58b49c0b937d28543d519e2
I20260811 05:17:36.032740 5996 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fcdfbd3a58b49c0b937d28543d519e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:36 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:40612
I20260811 05:17:36.032975 6428 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fcdfbd3a58b49c0b937d28543d519e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:36 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:57280
I20260811 05:17:36.033002 6211 tablet_service.cc:1514] Processing DeleteTablet for tablet 0fcdfbd3a58b49c0b937d28543d519e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:36 UTC) from {username='kudu', principal='kudu/127.7.67.190@KRBTEST.COM'} at 127.0.0.1:54974
I20260811 05:17:36.033330 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 5898
I20260811 05:17:36.041553 6607 tablet_replica.cc:330] T 0fcdfbd3a58b49c0b937d28543d519e2 P c04c5e7f9d4f480d94f0ef50ceb17171: stopping tablet replica
I20260811 05:17:36.041819 6606 tablet_replica.cc:330] T 0fcdfbd3a58b49c0b937d28543d519e2 P c0582d9d943c422fb5efb936515eb1a9: stopping tablet replica
I20260811 05:17:36.041975 6607 raft_consensus.cc:2229] T 0fcdfbd3a58b49c0b937d28543d519e2 P c04c5e7f9d4f480d94f0ef50ceb17171 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:17:36.042162 6606 raft_consensus.cc:2229] T 0fcdfbd3a58b49c0b937d28543d519e2 P c0582d9d943c422fb5efb936515eb1a9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:17:36.042353 6607 raft_consensus.cc:2258] T 0fcdfbd3a58b49c0b937d28543d519e2 P c04c5e7f9d4f480d94f0ef50ceb17171 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:36.042485 6606 raft_consensus.cc:2258] T 0fcdfbd3a58b49c0b937d28543d519e2 P c0582d9d943c422fb5efb936515eb1a9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:36.044872 6606 ts_tablet_manager.cc:1905] T 0fcdfbd3a58b49c0b937d28543d519e2 P c0582d9d943c422fb5efb936515eb1a9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:17:36.045542 6607 ts_tablet_manager.cc:1905] T 0fcdfbd3a58b49c0b937d28543d519e2 P c04c5e7f9d4f480d94f0ef50ceb17171: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:17:36.047350 6606 ts_tablet_manager.cc:1918] T 0fcdfbd3a58b49c0b937d28543d519e2 P c0582d9d943c422fb5efb936515eb1a9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:17:36.047464 6606 log.cc:1196] T 0fcdfbd3a58b49c0b937d28543d519e2 P c0582d9d943c422fb5efb936515eb1a9: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/0fcdfbd3a58b49c0b937d28543d519e2
I20260811 05:17:36.047806 6606 ts_tablet_manager.cc:1939] T 0fcdfbd3a58b49c0b937d28543d519e2 P c0582d9d943c422fb5efb936515eb1a9: Deleting consensus metadata
I20260811 05:17:36.049068 6607 ts_tablet_manager.cc:1918] T 0fcdfbd3a58b49c0b937d28543d519e2 P c04c5e7f9d4f480d94f0ef50ceb17171: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:17:36.049070 5774 catalog_manager.cc:4841] TS c0582d9d943c422fb5efb936515eb1a9 (127.7.67.131:34417): tablet 0fcdfbd3a58b49c0b937d28543d519e2 (table test-table [id=92c6d54960c243019f67d3aebb9afe7a]) successfully deleted
I20260811 05:17:36.049176 6607 log.cc:1196] T 0fcdfbd3a58b49c0b937d28543d519e2 P c04c5e7f9d4f480d94f0ef50ceb17171: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/0fcdfbd3a58b49c0b937d28543d519e2
I20260811 05:17:36.049495 6607 ts_tablet_manager.cc:1939] T 0fcdfbd3a58b49c0b937d28543d519e2 P c04c5e7f9d4f480d94f0ef50ceb17171: Deleting consensus metadata
I20260811 05:17:36.050830 5774 catalog_manager.cc:4841] TS c04c5e7f9d4f480d94f0ef50ceb17171 (127.7.67.130:33367): tablet 0fcdfbd3a58b49c0b937d28543d519e2 (table test-table [id=92c6d54960c243019f67d3aebb9afe7a]) successfully deleted
W20260811 05:17:36.053304 5772 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.129:35571 (error 108)
I20260811 05:17:36.053407 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 6117
I20260811 05:17:36.060849 6557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.059619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:35571 (local address 127.0.0.1:40636)
0811 05:17:36.060651 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.060736 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.129:35571: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":868}
W20260811 05:17:36.061079 5772 catalog_manager.cc:4584] TS 5e854a099d2548eb8a4dcbb2e4a8ee36 (127.7.67.129:35571): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0fcdfbd3a58b49c0b937d28543d519e2: Network error: Client connection negotiation failed: client connection to 127.7.67.129:35571: connect: Connection refused (error 111)
I20260811 05:17:36.066743 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 6327
I20260811 05:17:36.078814 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 5746
2026-08-11T05:17:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2644 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T05:17:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:36Z Disabled control of system clock
I20260811 05:17:36.118053 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:37995
--webserver_interface=127.7.67.190
--webserver_port=0
--builtin_ntp_servers=127.7.67.148:40133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:37995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 05:17:36.231338 6614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:36.231633 6614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:36.231709 6614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:36.235718 6614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:36.235790 6614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:36.235819 6614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:36.235865 6614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:36.235898 6614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:36.240573 6614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:40133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:37995
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:37995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:36.241653 6614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:36.243064 6614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:36.251421 6622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:36.251434 6621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:36.251519 6625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:36.251693 6614 server_base.cc:1034] running on GCE node
I20260811 05:17:36.252367 6614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:36.253496 6614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:36.254721 6614 hybrid_clock.cc:648] HybridClock initialized: now 1786425456254705 us; error 28 us; skew 500 ppm
I20260811 05:17:36.256716 6614 webserver.cc:460] Webserver started at http://127.7.67.190:40263/ using document root <none> and password file <none>
I20260811 05:17:36.257288 6614 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:36.257376 6614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:36.257603 6614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:36.259893 6614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "eca3e0f607984e1f9bcd5c038cae3add"
format_stamp: "Formatted at 2026-08-11 05:17:36 on dist-test-slave-04bb"
server_key: "7b12916972bf4c69c4133d1af7e76b88"
server_key_iv: "396c41187c0c9fe6caa0870b5fea8c58"
server_key_version: "encryptionkey@0"
I20260811 05:17:36.260516 6614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "eca3e0f607984e1f9bcd5c038cae3add"
format_stamp: "Formatted at 2026-08-11 05:17:36 on dist-test-slave-04bb"
server_key: "7b12916972bf4c69c4133d1af7e76b88"
server_key_iv: "396c41187c0c9fe6caa0870b5fea8c58"
server_key_version: "encryptionkey@0"
I20260811 05:17:36.264402 6614 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 05:17:36.267019 6633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:36.268338 6614 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:17:36.268477 6614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "eca3e0f607984e1f9bcd5c038cae3add"
format_stamp: "Formatted at 2026-08-11 05:17:36 on dist-test-slave-04bb"
server_key: "7b12916972bf4c69c4133d1af7e76b88"
server_key_iv: "396c41187c0c9fe6caa0870b5fea8c58"
server_key_version: "encryptionkey@0"
I20260811 05:17:36.268635 6614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:36.283861 6614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:36.284668 6614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:36.284878 6614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:36.292151 6614 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:37995
I20260811 05:17:36.292162 6723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:37995 every 8 connection(s)
I20260811 05:17:36.293211 6614 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:36.296481 6724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:36.299172 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 6614
I20260811 05:17:36.299315 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:17:36.299643 7438 external_mini_cluster.cc:1428] Setting key 5138bb4358956643ee391730ddcd41a2
I20260811 05:17:36.303264 6724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add: Bootstrap starting.
I20260811 05:17:36.306078 6724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:36.306965 6724 log.cc:824] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:36.309523 6724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add: No bootstrap required, opened a new log
I20260811 05:17:36.310520 6728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.301105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38990 (local address 127.7.67.190:37995)
0811 05:17:36.301766 (+ 661us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.301787 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.301841 (+ 54us) server_negotiation.cc:407] Connection header received
0811 05:17:36.302681 (+ 840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.302704 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.303195 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.303576 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.304033 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.305552 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.306534 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.307234 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.307416 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.308463 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:36.308469 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:36.308485 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:36.309033 (+ 548us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:36.309105 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:36.309131 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:36.309243 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:36.309323 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 05:17:36.309588 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":103,"threads_started":1}
I20260811 05:17:36.313607 6724 raft_consensus.cc:348] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca3e0f607984e1f9bcd5c038cae3add" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37995 } }
I20260811 05:17:36.313865 6724 raft_consensus.cc:374] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:36.313905 6724 raft_consensus.cc:729] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eca3e0f607984e1f9bcd5c038cae3add, State: Initialized, Role: FOLLOWER
I20260811 05:17:36.314309 6724 consensus_queue.cc:260] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca3e0f607984e1f9bcd5c038cae3add" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37995 } }
I20260811 05:17:36.314431 6724 raft_consensus.cc:388] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:36.314478 6724 raft_consensus.cc:482] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:36.314550 6724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:36.315433 6724 raft_consensus.cc:504] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca3e0f607984e1f9bcd5c038cae3add" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37995 } }
I20260811 05:17:36.315737 6724 leader_election.cc:302] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eca3e0f607984e1f9bcd5c038cae3add; no voters:
I20260811 05:17:36.316039 6724 leader_election.cc:288] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:36.316169 6730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:36.316397 6730 raft_consensus.cc:686] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add [term 1 LEADER]: Becoming Leader. State: Replica: eca3e0f607984e1f9bcd5c038cae3add, State: Running, Role: LEADER
I20260811 05:17:36.316745 6730 consensus_queue.cc:237] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca3e0f607984e1f9bcd5c038cae3add" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37995 } }
I20260811 05:17:36.317286 6724 sys_catalog.cc:564] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:36.318420 6735 sys_catalog.cc:455] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add [sys.catalog]: SysCatalogTable state changed. Reason: New leader eca3e0f607984e1f9bcd5c038cae3add. Latest consensus state: current_term: 1 leader_uuid: "eca3e0f607984e1f9bcd5c038cae3add" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca3e0f607984e1f9bcd5c038cae3add" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37995 } } }
I20260811 05:17:36.318559 6735 sys_catalog.cc:458] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:36.319063 6733 sys_catalog.cc:455] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eca3e0f607984e1f9bcd5c038cae3add" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eca3e0f607984e1f9bcd5c038cae3add" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37995 } } }
I20260811 05:17:36.319185 6733 sys_catalog.cc:458] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add [sys.catalog]: This master's current role is: LEADER
W20260811 05:17:36.322326 6756 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:17:36.322391 6756 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:17:36.322474 6744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:36.323158 6744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:36.328966 6744 catalog_manager.cc:1347] Generated new cluster ID: 4a934f53c3cd4c038871d45eaa9928a5
I20260811 05:17:36.329046 6744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:36.342046 6744 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:36.343317 6744 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:36.352975 6744 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add: Generated new TSK 0
I20260811 05:17:36.353765 6744 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:36.366652 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.129:0
--local_ip_for_outbound_sockets=127.7.67.129
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:37995
--builtin_ntp_servers=127.7.67.148:40133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 05:17:36.488775 6761 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:36.489056 6761 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:36.489121 6761 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:36.493222 6761 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:36.493289 6761 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:36.493369 6761 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.129
I20260811 05:17:36.498167 6761 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:40133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.7.67.129
--webserver_port=0
--tserver_master_addrs=127.7.67.190:37995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6761
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.129
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:36.499253 6761 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:36.500664 6761 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:36.510538 6770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:36.510586 6761 server_base.cc:1034] running on GCE node
W20260811 05:17:36.510531 6772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:36.510530 6769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:36.511263 6761 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:36.511847 6761 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:36.513007 6761 hybrid_clock.cc:648] HybridClock initialized: now 1786425456512992 us; error 26 us; skew 500 ppm
I20260811 05:17:36.514895 6761 webserver.cc:460] Webserver started at http://127.7.67.129:33215/ using document root <none> and password file <none>
I20260811 05:17:36.515494 6761 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:36.515558 6761 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:36.515781 6761 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:36.517985 6761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-0/data/instance:
uuid: "264006b018974602a619e40f953558f9"
format_stamp: "Formatted at 2026-08-11 05:17:36 on dist-test-slave-04bb"
server_key: "b7854d50cf8aec135e2e65c15a1fd1db"
server_key_iv: "9b16342dbf367ee0cfecedf5987541ea"
server_key_version: "encryptionkey@0"
I20260811 05:17:36.518574 6761 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance:
uuid: "264006b018974602a619e40f953558f9"
format_stamp: "Formatted at 2026-08-11 05:17:36 on dist-test-slave-04bb"
server_key: "b7854d50cf8aec135e2e65c15a1fd1db"
server_key_iv: "9b16342dbf367ee0cfecedf5987541ea"
server_key_version: "encryptionkey@0"
I20260811 05:17:36.522305 6761 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 05:17:36.524672 6782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:36.525946 6761 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:17:36.526087 6761 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-0/wal
uuid: "264006b018974602a619e40f953558f9"
format_stamp: "Formatted at 2026-08-11 05:17:36 on dist-test-slave-04bb"
server_key: "b7854d50cf8aec135e2e65c15a1fd1db"
server_key_iv: "9b16342dbf367ee0cfecedf5987541ea"
server_key_version: "encryptionkey@0"
I20260811 05:17:36.526187 6761 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:36.546267 6761 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:36.546893 6761 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:36.547076 6761 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:36.547634 6761 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:36.548616 6761 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:36.548666 6761 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:36.548725 6761 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:36.548756 6761 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:36.558032 6761 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.129:42315
I20260811 05:17:36.558063 6954 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.129:42315 every 8 connection(s)
I20260811 05:17:36.558943 6761 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-0/data/info.pb
I20260811 05:17:36.564914 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 6761
I20260811 05:17:36.565052 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-0/wal/instance
I20260811 05:17:36.565375 7438 external_mini_cluster.cc:1428] Setting key 9daf677ae5a0c63974044feb7035fbf1
I20260811 05:17:36.567485 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.130:0
--local_ip_for_outbound_sockets=127.7.67.130
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:37995
--builtin_ntp_servers=127.7.67.148:40133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 05:17:36.569248 6728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.560971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.129:36515 (local address 127.7.67.190:37995)
0811 05:17:36.561128 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.561131 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.562273 (+ 1142us) server_negotiation.cc:407] Connection header received
0811 05:17:36.563734 (+ 1461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.563738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.563798 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.563896 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.564898 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.565572 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.567161 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.567395 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.567504 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.568751 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:36.568755 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:36.568758 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:36.568771 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:36.568797 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:36.568806 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:36.568882 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:36.569043 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 05:17:36.569108 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:17:36.570169 6958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.561414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37995 (local address 127.7.67.129:36515)
0811 05:17:36.562071 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.562111 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:36.563501 (+ 1390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:36.564072 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:36.564079 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:36.564161 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:36.564745 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.564760 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.565697 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.565706 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.567036 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.567046 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.567546 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.567551 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.567761 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.568430 (+ 669us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:17:36.568432 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:36.568472 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:17:36.568586 (+ 114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:36.568885 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:36.568897 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:36.568906 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 05:17:36.569042 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 05:17:36.569334 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":104,"threads_started":1}
I20260811 05:17:36.571416 6956 heartbeater.cc:347] Connected to a master server at 127.7.67.190:37995
I20260811 05:17:36.571650 6956 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:36.572160 6956 heartbeater.cc:511] Master 127.7.67.190:37995 requested a full tablet report, sending...
I20260811 05:17:36.573966 6667 ts_manager.cc:194] Registered new tserver with Master: 264006b018974602a619e40f953558f9 (127.7.67.129:42315)
I20260811 05:17:36.575721 6667 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.67.129:36515
W20260811 05:17:36.688573 6959 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:36.688872 6959 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:36.688911 6959 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:36.692979 6959 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:36.693029 6959 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:36.693082 6959 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.130
I20260811 05:17:36.697804 6959 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:40133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.7.67.130
--webserver_port=0
--tserver_master_addrs=127.7.67.190:37995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6959
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.130
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:36.698823 6959 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:36.700166 6959 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:36.710104 6968 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:36.710156 6966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:36.710170 6972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:36.710511 6959 server_base.cc:1034] running on GCE node
I20260811 05:17:36.710963 6959 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:36.711678 6959 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:36.712843 6959 hybrid_clock.cc:648] HybridClock initialized: now 1786425456712827 us; error 25 us; skew 500 ppm
I20260811 05:17:36.714747 6959 webserver.cc:460] Webserver started at http://127.7.67.130:46209/ using document root <none> and password file <none>
I20260811 05:17:36.715390 6959 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:36.715484 6959 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:36.715724 6959 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:36.717995 6959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-1/data/instance:
uuid: "fbeea6ac75b940f5bc9728abf4f50810"
format_stamp: "Formatted at 2026-08-11 05:17:36 on dist-test-slave-04bb"
server_key: "64be4c0d247631a93267d76e2bd96fff"
server_key_iv: "e7e240530289c7dedf53cb49f55af93a"
server_key_version: "encryptionkey@0"
I20260811 05:17:36.718568 6959 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance:
uuid: "fbeea6ac75b940f5bc9728abf4f50810"
format_stamp: "Formatted at 2026-08-11 05:17:36 on dist-test-slave-04bb"
server_key: "64be4c0d247631a93267d76e2bd96fff"
server_key_iv: "e7e240530289c7dedf53cb49f55af93a"
server_key_version: "encryptionkey@0"
I20260811 05:17:36.722368 6959 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260811 05:17:36.724829 6982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:36.726064 6959 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:17:36.726194 6959 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-1/wal
uuid: "fbeea6ac75b940f5bc9728abf4f50810"
format_stamp: "Formatted at 2026-08-11 05:17:36 on dist-test-slave-04bb"
server_key: "64be4c0d247631a93267d76e2bd96fff"
server_key_iv: "e7e240530289c7dedf53cb49f55af93a"
server_key_version: "encryptionkey@0"
I20260811 05:17:36.726286 6959 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:36.740103 6959 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:36.740849 6959 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:36.741045 6959 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:36.741623 6959 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:36.742568 6959 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:36.742638 6959 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:36.742707 6959 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:36.742746 6959 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:36.751534 6959 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.130:46759
I20260811 05:17:36.751567 7152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.130:46759 every 8 connection(s)
I20260811 05:17:36.752429 6959 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-1/data/info.pb
I20260811 05:17:36.755610 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 6959
I20260811 05:17:36.755752 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-1/wal/instance
I20260811 05:17:36.756090 7438 external_mini_cluster.cc:1428] Setting key 4e9466270e5c1b83184dfd4401f345d5
I20260811 05:17:36.758366 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.7.67.131:0
--local_ip_for_outbound_sockets=127.7.67.131
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:37995
--builtin_ntp_servers=127.7.67.148:40133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 05:17:36.762969 6728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.754290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:58299 (local address 127.7.67.190:37995)
0811 05:17:36.754430 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.754434 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.755368 (+ 934us) server_negotiation.cc:407] Connection header received
0811 05:17:36.757274 (+ 1906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.757278 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.757331 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.757412 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.758590 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.759428 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.761010 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.761239 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.761352 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.762467 (+ 1115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:36.762471 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:36.762474 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:36.762493 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:36.762512 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:36.762520 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:36.762598 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:36.762757 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 05:17:36.762821 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 05:17:36.763797 7156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.754580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37995 (local address 127.7.67.130:58299)
0811 05:17:36.755200 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.755240 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:36.757070 (+ 1830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:36.757580 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:36.757587 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:36.757709 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:36.758398 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.758423 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.759607 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.759613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.760878 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.760889 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.761393 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.761398 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.761571 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.762204 (+ 633us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:17:36.762206 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:36.762248 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:17:36.762311 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:36.762601 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:36.762612 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:36.762620 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 05:17:36.762756 (+ 136us) client_negotiation.cc:240] Negotiation successful
0811 05:17:36.763032 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":101,"threads_started":1}
I20260811 05:17:36.764935 7153 heartbeater.cc:347] Connected to a master server at 127.7.67.190:37995
I20260811 05:17:36.765194 7153 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:36.765708 7153 heartbeater.cc:511] Master 127.7.67.190:37995 requested a full tablet report, sending...
I20260811 05:17:36.766762 6667 ts_manager.cc:194] Registered new tserver with Master: fbeea6ac75b940f5bc9728abf4f50810 (127.7.67.130:46759)
I20260811 05:17:36.767333 6667 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.67.130:58299
W20260811 05:17:36.875356 7157 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:36.875633 7157 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:36.875702 7157 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:36.879635 7157 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:36.879714 7157 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:36.879803 7157 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.7.67.131
I20260811 05:17:36.884508 7157 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:40133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.7.67.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.7.67.131
--webserver_port=0
--tserver_master_addrs=127.7.67.190:37995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7157
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.7.67.131
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:36.885627 7157 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:36.887017 7157 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:36.896908 7166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:36.896901 7163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:36.896911 7164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:36.897243 7157 server_base.cc:1034] running on GCE node
I20260811 05:17:36.897708 7157 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:36.898370 7157 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:36.899559 7157 hybrid_clock.cc:648] HybridClock initialized: now 1786425456899536 us; error 33 us; skew 500 ppm
I20260811 05:17:36.901536 7157 webserver.cc:460] Webserver started at http://127.7.67.131:38467/ using document root <none> and password file <none>
I20260811 05:17:36.902143 7157 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:36.902236 7157 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:36.902477 7157 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:36.904634 7157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-2/data/instance:
uuid: "393e7cf0ccab4eab95c16d5d51630051"
format_stamp: "Formatted at 2026-08-11 05:17:36 on dist-test-slave-04bb"
server_key: "d5f2fe2b858d204341aae2d6acfdc697"
server_key_iv: "02d9e21901037d6c773bc06f5cdfc918"
server_key_version: "encryptionkey@0"
I20260811 05:17:36.905210 7157 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance:
uuid: "393e7cf0ccab4eab95c16d5d51630051"
format_stamp: "Formatted at 2026-08-11 05:17:36 on dist-test-slave-04bb"
server_key: "d5f2fe2b858d204341aae2d6acfdc697"
server_key_iv: "02d9e21901037d6c773bc06f5cdfc918"
server_key_version: "encryptionkey@0"
I20260811 05:17:36.909118 7157 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.002s
I20260811 05:17:36.911580 7175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:36.912894 7157 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:17:36.913033 7157 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-2/wal
uuid: "393e7cf0ccab4eab95c16d5d51630051"
format_stamp: "Formatted at 2026-08-11 05:17:36 on dist-test-slave-04bb"
server_key: "d5f2fe2b858d204341aae2d6acfdc697"
server_key_iv: "02d9e21901037d6c773bc06f5cdfc918"
server_key_version: "encryptionkey@0"
I20260811 05:17:36.913148 7157 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:36.922852 7157 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:36.923508 7157 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:36.923697 7157 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:36.924268 7157 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 05:17:36.925292 7157 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 05:17:36.925360 7157 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:36.925429 7157 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 05:17:36.925468 7157 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:36.934620 7157 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.131:38975
I20260811 05:17:36.934655 7354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.131:38975 every 8 connection(s)
I20260811 05:17:36.935516 7157 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-2/data/info.pb
I20260811 05:17:36.935865 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 7157
I20260811 05:17:36.935992 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-2/wal/instance
I20260811 05:17:36.936345 7438 external_mini_cluster.cc:1428] Setting key ffd8d401afa70a696b80c8fc86d7ecbd
I20260811 05:17:36.944638 6728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.937368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.131:52629 (local address 127.7.67.190:37995)
0811 05:17:36.937507 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.937511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.938880 (+ 1369us) server_negotiation.cc:407] Connection header received
0811 05:17:36.940255 (+ 1375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.940257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.940305 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.940365 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.941205 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.941713 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.942830 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.943048 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.943125 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.944254 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:36.944257 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:36.944259 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:36.944270 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:36.944284 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:36.944291 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:36.944343 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:36.944457 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 05:17:36.944505 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 05:17:36.945557 7358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.937940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37995 (local address 127.7.67.131:52629)
0811 05:17:36.938703 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.938765 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:36.940059 (+ 1294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:36.940496 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:36.940503 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:36.940604 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:17:36.941087 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.941099 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.941841 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.941843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.942724 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.942733 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.943176 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.943178 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.943341 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.944021 (+ 680us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 05:17:36.944022 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:17:36.944064 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 05:17:36.944126 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:17:36.944350 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:17:36.944354 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:17:36.944358 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 05:17:36.944446 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 05:17:36.944744 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":109,"threads_started":1}
I20260811 05:17:36.946686 7355 heartbeater.cc:347] Connected to a master server at 127.7.67.190:37995
I20260811 05:17:36.946890 7355 heartbeater.cc:464] Registering TS with master...
I20260811 05:17:36.947360 7355 heartbeater.cc:511] Master 127.7.67.190:37995 requested a full tablet report, sending...
I20260811 05:17:36.948388 6667 ts_manager.cc:194] Registered new tserver with Master: 393e7cf0ccab4eab95c16d5d51630051 (127.7.67.131:38975)
I20260811 05:17:36.949044 6667 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.7.67.131:52629
I20260811 05:17:36.950697 7438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 05:17:36.956583 6728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.952725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38996 (local address 127.7.67.190:37995)
0811 05:17:36.952850 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.952853 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.953072 (+ 219us) server_negotiation.cc:407] Connection header received
0811 05:17:36.953216 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.953218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.953254 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.953338 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:36.953581 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.954058 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.954931 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.955139 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.955217 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.956078 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:36.956083 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:36.956085 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:36.956097 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:36.956117 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:36.956125 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:36.956186 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:36.956374 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 05:17:36.956428 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 05:17:36.959553 6667 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38996:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 05:17:36.962671 6667 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 05:17:36.977506 7375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.971559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47548 (local address 127.7.67.130:46759)
0811 05:17:36.971922 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.971935 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.972592 (+ 657us) server_negotiation.cc:407] Connection header received
0811 05:17:36.972683 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.972689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.972911 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.973045 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:36.973429 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.974450 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.975963 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.976788 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.976915 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.977122 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:36.977232 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 05:17:36.977331 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":134,"threads_started":1}
I20260811 05:17:36.977697 7376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.972172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:38975 (local address 127.0.0.1:42990)
0811 05:17:36.972903 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.972916 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:36.973011 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:36.973455 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:36.973457 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:36.973477 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:36.973568 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.973573 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.975011 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.975015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.976094 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.976101 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.977229 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.977233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.977325 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.977359 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 05:17:36.977439 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 05:17:36.977519 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":28,"thread_start_us":93,"threads_started":1}
I20260811 05:17:36.977945 7374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.971533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:42315 (local address 127.0.0.1:44396)
0811 05:17:36.972606 (+ 1073us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.972620 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:36.972696 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:36.973789 (+ 1093us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:36.973792 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:36.973810 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:36.973888 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.973893 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.975356 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.975361 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.976545 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.976558 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.977504 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.977507 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.977595 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.977698 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 05:17:36.977753 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 05:17:36.977829 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1007,"thread_start_us":107,"threads_started":1}
I20260811 05:17:36.977499 7373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.971433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.130:46759 (local address 127.0.0.1:47548)
0811 05:17:36.972215 (+ 782us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:36.972299 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:36.972440 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:36.973161 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:36.973167 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:36.973191 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:36.973307 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.973312 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.974552 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.974558 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.975848 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:36.975859 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.976928 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.976931 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:36.977038 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.977079 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 05:17:36.977202 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 05:17:36.977340 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"mutex_wait_us":3,"thread_start_us":91,"threads_started":1}
I20260811 05:17:36.978292 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.972357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42990 (local address 127.7.67.131:38975)
0811 05:17:36.972889 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.972901 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.973007 (+ 106us) server_negotiation.cc:407] Connection header received
0811 05:17:36.973123 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.973128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.973315 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.973464 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:36.973659 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.974885 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.976202 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.976888 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.976987 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.977162 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:36.978029 (+ 867us) server_negotiation.cc:299] Negotiation successful
0811 05:17:36.978134 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":184,"threads_started":1}
I20260811 05:17:36.978703 7377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:36.972280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44396 (local address 127.7.67.129:42315)
0811 05:17:36.973280 (+ 1000us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:36.973292 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:36.973326 (+ 34us) server_negotiation.cc:407] Connection header received
0811 05:17:36.973403 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:36.973411 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:36.973637 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:36.973793 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:36.974060 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.975202 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.976648 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:36.977377 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:36.978217 (+ 840us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:36.978415 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:36.978467 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 05:17:36.978555 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":899,"thread_start_us":128,"threads_started":1}
I20260811 05:17:36.981304 6853 tablet_service.cc:1467] Processing CreateTablet for tablet d49b5f45d06f415da0529ce2f6befc5d (DEFAULT_TABLE table=test-table [id=e22d9c5a742c4516946306a7cc569f79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:36.981316 7054 tablet_service.cc:1467] Processing CreateTablet for tablet d49b5f45d06f415da0529ce2f6befc5d (DEFAULT_TABLE table=test-table [id=e22d9c5a742c4516946306a7cc569f79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:36.981330 7255 tablet_service.cc:1467] Processing CreateTablet for tablet d49b5f45d06f415da0529ce2f6befc5d (DEFAULT_TABLE table=test-table [id=e22d9c5a742c4516946306a7cc569f79]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 05:17:36.982308 6853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d49b5f45d06f415da0529ce2f6befc5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:36.982363 7054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d49b5f45d06f415da0529ce2f6befc5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:36.982363 7255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d49b5f45d06f415da0529ce2f6befc5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:36.989076 7381 tablet_bootstrap.cc:492] T d49b5f45d06f415da0529ce2f6befc5d P 264006b018974602a619e40f953558f9: Bootstrap starting.
I20260811 05:17:36.990165 7383 tablet_bootstrap.cc:492] T d49b5f45d06f415da0529ce2f6befc5d P 393e7cf0ccab4eab95c16d5d51630051: Bootstrap starting.
I20260811 05:17:36.991400 7381 tablet_bootstrap.cc:654] T d49b5f45d06f415da0529ce2f6befc5d P 264006b018974602a619e40f953558f9: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:36.992007 7383 tablet_bootstrap.cc:654] T d49b5f45d06f415da0529ce2f6befc5d P 393e7cf0ccab4eab95c16d5d51630051: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:36.992180 7381 log.cc:824] T d49b5f45d06f415da0529ce2f6befc5d P 264006b018974602a619e40f953558f9: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:36.992847 7382 tablet_bootstrap.cc:492] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810: Bootstrap starting.
I20260811 05:17:36.992903 7383 log.cc:824] T d49b5f45d06f415da0529ce2f6befc5d P 393e7cf0ccab4eab95c16d5d51630051: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:36.994498 7381 tablet_bootstrap.cc:492] T d49b5f45d06f415da0529ce2f6befc5d P 264006b018974602a619e40f953558f9: No bootstrap required, opened a new log
I20260811 05:17:36.994699 7381 ts_tablet_manager.cc:1397] T d49b5f45d06f415da0529ce2f6befc5d P 264006b018974602a619e40f953558f9: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.002s
I20260811 05:17:36.994957 7382 tablet_bootstrap.cc:654] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:36.995038 7383 tablet_bootstrap.cc:492] T d49b5f45d06f415da0529ce2f6befc5d P 393e7cf0ccab4eab95c16d5d51630051: No bootstrap required, opened a new log
I20260811 05:17:36.995215 7383 ts_tablet_manager.cc:1397] T d49b5f45d06f415da0529ce2f6befc5d P 393e7cf0ccab4eab95c16d5d51630051: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:17:36.995781 7382 log.cc:824] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:36.997761 7382 tablet_bootstrap.cc:492] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810: No bootstrap required, opened a new log
I20260811 05:17:36.997953 7382 ts_tablet_manager.cc:1397] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 05:17:36.998649 7383 raft_consensus.cc:348] T d49b5f45d06f415da0529ce2f6befc5d P 393e7cf0ccab4eab95c16d5d51630051 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbeea6ac75b940f5bc9728abf4f50810" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 46759 } } peers { permanent_uuid: "264006b018974602a619e40f953558f9" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42315 } } peers { permanent_uuid: "393e7cf0ccab4eab95c16d5d51630051" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38975 } }
I20260811 05:17:36.998891 7383 raft_consensus.cc:374] T d49b5f45d06f415da0529ce2f6befc5d P 393e7cf0ccab4eab95c16d5d51630051 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:36.998946 7383 raft_consensus.cc:729] T d49b5f45d06f415da0529ce2f6befc5d P 393e7cf0ccab4eab95c16d5d51630051 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 393e7cf0ccab4eab95c16d5d51630051, State: Initialized, Role: FOLLOWER
I20260811 05:17:36.999009 7381 raft_consensus.cc:348] T d49b5f45d06f415da0529ce2f6befc5d P 264006b018974602a619e40f953558f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbeea6ac75b940f5bc9728abf4f50810" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 46759 } } peers { permanent_uuid: "264006b018974602a619e40f953558f9" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42315 } } peers { permanent_uuid: "393e7cf0ccab4eab95c16d5d51630051" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38975 } }
I20260811 05:17:36.999264 7381 raft_consensus.cc:374] T d49b5f45d06f415da0529ce2f6befc5d P 264006b018974602a619e40f953558f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:36.999333 7381 raft_consensus.cc:729] T d49b5f45d06f415da0529ce2f6befc5d P 264006b018974602a619e40f953558f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 264006b018974602a619e40f953558f9, State: Initialized, Role: FOLLOWER
I20260811 05:17:36.999447 7383 consensus_queue.cc:260] T d49b5f45d06f415da0529ce2f6befc5d P 393e7cf0ccab4eab95c16d5d51630051 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbeea6ac75b940f5bc9728abf4f50810" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 46759 } } peers { permanent_uuid: "264006b018974602a619e40f953558f9" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42315 } } peers { permanent_uuid: "393e7cf0ccab4eab95c16d5d51630051" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38975 } }
I20260811 05:17:36.999754 7381 consensus_queue.cc:260] T d49b5f45d06f415da0529ce2f6befc5d P 264006b018974602a619e40f953558f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbeea6ac75b940f5bc9728abf4f50810" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 46759 } } peers { permanent_uuid: "264006b018974602a619e40f953558f9" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42315 } } peers { permanent_uuid: "393e7cf0ccab4eab95c16d5d51630051" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38975 } }
I20260811 05:17:37.000208 7355 heartbeater.cc:503] Master 127.7.67.190:37995 was elected leader, sending a full tablet report...
I20260811 05:17:37.000802 7381 ts_tablet_manager.cc:1428] T d49b5f45d06f415da0529ce2f6befc5d P 264006b018974602a619e40f953558f9: Time spent starting tablet: real 0.006s user 0.004s sys 0.003s
I20260811 05:17:37.001236 6956 heartbeater.cc:503] Master 127.7.67.190:37995 was elected leader, sending a full tablet report...
I20260811 05:17:37.001976 7383 ts_tablet_manager.cc:1428] T d49b5f45d06f415da0529ce2f6befc5d P 393e7cf0ccab4eab95c16d5d51630051: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 05:17:37.002085 7382 raft_consensus.cc:348] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbeea6ac75b940f5bc9728abf4f50810" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 46759 } } peers { permanent_uuid: "264006b018974602a619e40f953558f9" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42315 } } peers { permanent_uuid: "393e7cf0ccab4eab95c16d5d51630051" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38975 } }
I20260811 05:17:37.002640 7382 raft_consensus.cc:374] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:37.002697 7382 raft_consensus.cc:729] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbeea6ac75b940f5bc9728abf4f50810, State: Initialized, Role: FOLLOWER
I20260811 05:17:37.003132 7382 consensus_queue.cc:260] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbeea6ac75b940f5bc9728abf4f50810" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 46759 } } peers { permanent_uuid: "264006b018974602a619e40f953558f9" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42315 } } peers { permanent_uuid: "393e7cf0ccab4eab95c16d5d51630051" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38975 } }
I20260811 05:17:37.003952 7153 heartbeater.cc:503] Master 127.7.67.190:37995 was elected leader, sending a full tablet report...
I20260811 05:17:37.004105 7382 ts_tablet_manager.cc:1428] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260811 05:17:37.060176 6957 tablet.cc:2369] T d49b5f45d06f415da0529ce2f6befc5d P 264006b018974602a619e40f953558f9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:17:37.063596 7392 raft_consensus.cc:482] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 05:17:37.063820 7392 raft_consensus.cc:504] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbeea6ac75b940f5bc9728abf4f50810" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 46759 } } peers { permanent_uuid: "264006b018974602a619e40f953558f9" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42315 } } peers { permanent_uuid: "393e7cf0ccab4eab95c16d5d51630051" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38975 } }
I20260811 05:17:37.065137 7392 leader_election.cc:288] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 264006b018974602a619e40f953558f9 (127.7.67.129:42315), 393e7cf0ccab4eab95c16d5d51630051 (127.7.67.131:38975)
I20260811 05:17:37.069758 7156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:37.065372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:42315 (local address 127.7.67.130:39083)
0811 05:17:37.065542 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:37.065559 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:37.065686 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:37.066094 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:37.066101 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:37.066126 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:37.066264 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:37.066275 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:37.067154 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:37.067158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:37.068223 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:37.068235 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:37.069280 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:37.069284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:37.069400 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:37.069436 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 05:17:37.069528 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 05:17:37.069604 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":52}
I20260811 05:17:37.069756 7377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:37.065459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:39083 (local address 127.7.67.129:42315)
0811 05:17:37.065603 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:37.065610 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:37.065630 (+ 20us) server_negotiation.cc:407] Connection header received
0811 05:17:37.065864 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:37.065868 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:37.065933 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:37.066026 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:37.066398 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:37.067030 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:37.068362 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:37.069042 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:37.069194 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:37.069267 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:37.069553 (+ 286us) server_negotiation.cc:299] Negotiation successful
0811 05:17:37.069641 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 05:17:37.069993 7395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:37.065520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.131:38975 (local address 127.7.67.130:45981)
0811 05:17:37.065918 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:37.065933 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:17:37.066038 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:17:37.066390 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:17:37.066394 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:17:37.066413 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 05:17:37.066529 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:37.066535 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:37.067423 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:37.067426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:37.068392 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:17:37.068403 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:17:37.069488 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:17:37.069495 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:17:37.069651 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:37.069674 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 05:17:37.069766 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 05:17:37.069844 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"mutex_wait_us":19,"thread_start_us":155,"threads_started":1}
I20260811 05:17:37.069978 7380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:37.065698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.7.67.130:45981 (local address 127.7.67.131:38975)
0811 05:17:37.065922 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:37.065926 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:37.066020 (+ 94us) server_negotiation.cc:407] Connection header received
0811 05:17:37.066193 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:37.066197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:37.066257 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:37.066355 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 05:17:37.066649 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:37.067307 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:37.068555 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:37.069190 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:37.069317 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:37.069374 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:37.069797 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 05:17:37.069875 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260811 05:17:37.070693 6887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d49b5f45d06f415da0529ce2f6befc5d" candidate_uuid: "fbeea6ac75b940f5bc9728abf4f50810" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "264006b018974602a619e40f953558f9" is_pre_election: true
I20260811 05:17:37.070731 7286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d49b5f45d06f415da0529ce2f6befc5d" candidate_uuid: "fbeea6ac75b940f5bc9728abf4f50810" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "393e7cf0ccab4eab95c16d5d51630051" is_pre_election: true
I20260811 05:17:37.071075 7286 raft_consensus.cc:2454] T d49b5f45d06f415da0529ce2f6befc5d P 393e7cf0ccab4eab95c16d5d51630051 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbeea6ac75b940f5bc9728abf4f50810 in term 0.
I20260811 05:17:37.071074 6887 raft_consensus.cc:2454] T d49b5f45d06f415da0529ce2f6befc5d P 264006b018974602a619e40f953558f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbeea6ac75b940f5bc9728abf4f50810 in term 0.
I20260811 05:17:37.071579 6986 leader_election.cc:302] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 393e7cf0ccab4eab95c16d5d51630051, fbeea6ac75b940f5bc9728abf4f50810; no voters:
I20260811 05:17:37.071887 7392 raft_consensus.cc:2781] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 05:17:37.071986 7392 raft_consensus.cc:482] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 05:17:37.072043 7392 raft_consensus.cc:3037] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:37.073393 7392 raft_consensus.cc:504] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbeea6ac75b940f5bc9728abf4f50810" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 46759 } } peers { permanent_uuid: "264006b018974602a619e40f953558f9" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42315 } } peers { permanent_uuid: "393e7cf0ccab4eab95c16d5d51630051" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38975 } }
I20260811 05:17:37.073777 7392 leader_election.cc:288] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810 [CANDIDATE]: Term 1 election: Requested vote from peers 264006b018974602a619e40f953558f9 (127.7.67.129:42315), 393e7cf0ccab4eab95c16d5d51630051 (127.7.67.131:38975)
I20260811 05:17:37.074168 7286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d49b5f45d06f415da0529ce2f6befc5d" candidate_uuid: "fbeea6ac75b940f5bc9728abf4f50810" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "393e7cf0ccab4eab95c16d5d51630051"
I20260811 05:17:37.074174 6887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d49b5f45d06f415da0529ce2f6befc5d" candidate_uuid: "fbeea6ac75b940f5bc9728abf4f50810" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "264006b018974602a619e40f953558f9"
I20260811 05:17:37.074304 6887 raft_consensus.cc:3037] T d49b5f45d06f415da0529ce2f6befc5d P 264006b018974602a619e40f953558f9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:37.074307 7286 raft_consensus.cc:3037] T d49b5f45d06f415da0529ce2f6befc5d P 393e7cf0ccab4eab95c16d5d51630051 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:37.075634 6887 raft_consensus.cc:2454] T d49b5f45d06f415da0529ce2f6befc5d P 264006b018974602a619e40f953558f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbeea6ac75b940f5bc9728abf4f50810 in term 1.
I20260811 05:17:37.075634 7286 raft_consensus.cc:2454] T d49b5f45d06f415da0529ce2f6befc5d P 393e7cf0ccab4eab95c16d5d51630051 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbeea6ac75b940f5bc9728abf4f50810 in term 1.
I20260811 05:17:37.075953 6990 leader_election.cc:302] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 264006b018974602a619e40f953558f9, fbeea6ac75b940f5bc9728abf4f50810; no voters:
I20260811 05:17:37.076117 7392 raft_consensus.cc:2781] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:37.076359 7392 raft_consensus.cc:686] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810 [term 1 LEADER]: Becoming Leader. State: Replica: fbeea6ac75b940f5bc9728abf4f50810, State: Running, Role: LEADER
I20260811 05:17:37.076748 7392 consensus_queue.cc:237] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbeea6ac75b940f5bc9728abf4f50810" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 46759 } } peers { permanent_uuid: "264006b018974602a619e40f953558f9" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42315 } } peers { permanent_uuid: "393e7cf0ccab4eab95c16d5d51630051" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38975 } }
I20260811 05:17:37.079939 6667 catalog_manager.cc:5495] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810 reported cstate change: term changed from 0 to 1, leader changed from <none> to fbeea6ac75b940f5bc9728abf4f50810 (127.7.67.130). New cstate: current_term: 1 leader_uuid: "fbeea6ac75b940f5bc9728abf4f50810" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbeea6ac75b940f5bc9728abf4f50810" member_type: VOTER last_known_addr { host: "127.7.67.130" port: 46759 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "264006b018974602a619e40f953558f9" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "393e7cf0ccab4eab95c16d5d51630051" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38975 } health_report { overall_health: UNKNOWN } } }
I20260811 05:17:37.101115 7375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:37.097631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47560 (local address 127.7.67.130:46759)
0811 05:17:37.097756 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:37.097759 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:37.097833 (+ 74us) server_negotiation.cc:407] Connection header received
0811 05:17:37.097977 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:37.097979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:37.098032 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:37.098120 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:17:37.098331 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:37.098826 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:37.099710 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:37.099921 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:37.099997 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:37.100253 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:17:37.100708 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:17:37.100813 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:37.100951 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 05:17:37.100995 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 05:17:37.106910 7286 raft_consensus.cc:1261] T d49b5f45d06f415da0529ce2f6befc5d P 393e7cf0ccab4eab95c16d5d51630051 [term 1 FOLLOWER]: Refusing update from remote peer fbeea6ac75b940f5bc9728abf4f50810: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:37.106938 6887 raft_consensus.cc:1261] T d49b5f45d06f415da0529ce2f6befc5d P 264006b018974602a619e40f953558f9 [term 1 FOLLOWER]: Refusing update from remote peer fbeea6ac75b940f5bc9728abf4f50810: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:17:37.107640 7399 consensus_queue.cc:1035] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810 [LEADER]: Connected to new peer: Peer: permanent_uuid: "393e7cf0ccab4eab95c16d5d51630051" member_type: VOTER last_known_addr { host: "127.7.67.131" port: 38975 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:37.107877 7392 consensus_queue.cc:1035] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810 [LEADER]: Connected to new peer: Peer: permanent_uuid: "264006b018974602a619e40f953558f9" member_type: VOTER last_known_addr { host: "127.7.67.129" port: 42315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:17:37.123843 6667 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:17:37.124060 6667 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 05:17:37.126801 6667 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eca3e0f607984e1f9bcd5c038cae3add: Sending DeleteTablet for 3 replicas of tablet d49b5f45d06f415da0529ce2f6befc5d
I20260811 05:17:37.127638 6853 tablet_service.cc:1514] Processing DeleteTablet for tablet d49b5f45d06f415da0529ce2f6befc5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:37 UTC) from {username='slave'} at 127.0.0.1:44396
I20260811 05:17:37.127744 7255 tablet_service.cc:1514] Processing DeleteTablet for tablet d49b5f45d06f415da0529ce2f6befc5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:37 UTC) from {username='slave'} at 127.0.0.1:42990
I20260811 05:17:37.128248 7421 tablet_replica.cc:330] T d49b5f45d06f415da0529ce2f6befc5d P 393e7cf0ccab4eab95c16d5d51630051: stopping tablet replica
I20260811 05:17:37.128347 7054 tablet_service.cc:1514] Processing DeleteTablet for tablet d49b5f45d06f415da0529ce2f6befc5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 05:17:37 UTC) from {username='slave'} at 127.0.0.1:47548
I20260811 05:17:37.128743 7421 raft_consensus.cc:2229] T d49b5f45d06f415da0529ce2f6befc5d P 393e7cf0ccab4eab95c16d5d51630051 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 05:17:37.129101 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 6761
I20260811 05:17:37.129108 7421 raft_consensus.cc:2258] T d49b5f45d06f415da0529ce2f6befc5d P 393e7cf0ccab4eab95c16d5d51630051 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:37.142127 7422 tablet_replica.cc:330] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810: stopping tablet replica
I20260811 05:17:37.142607 7422 raft_consensus.cc:2229] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810 [term 1 LEADER]: Raft consensus shutting down.
I20260811 05:17:37.143100 7422 raft_consensus.cc:2258] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 05:17:37.144945 7422 ts_tablet_manager.cc:1905] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:17:37.145220 7421 ts_tablet_manager.cc:1905] T d49b5f45d06f415da0529ce2f6befc5d P 393e7cf0ccab4eab95c16d5d51630051: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 05:17:37.148141 7422 ts_tablet_manager.cc:1918] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:17:37.148267 7422 log.cc:1196] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-1/wal/wals/d49b5f45d06f415da0529ce2f6befc5d
I20260811 05:17:37.148695 7422 ts_tablet_manager.cc:1939] T d49b5f45d06f415da0529ce2f6befc5d P fbeea6ac75b940f5bc9728abf4f50810: Deleting consensus metadata
I20260811 05:17:37.148746 7421 ts_tablet_manager.cc:1918] T d49b5f45d06f415da0529ce2f6befc5d P 393e7cf0ccab4eab95c16d5d51630051: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 05:17:37.148846 7421 log.cc:1196] T d49b5f45d06f415da0529ce2f6befc5d P 393e7cf0ccab4eab95c16d5d51630051: Deleting WAL directory at /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786425190664461-7438-0/minicluster-data/ts-2/wal/wals/d49b5f45d06f415da0529ce2f6befc5d
I20260811 05:17:37.149197 7421 ts_tablet_manager.cc:1939] T d49b5f45d06f415da0529ce2f6befc5d P 393e7cf0ccab4eab95c16d5d51630051: Deleting consensus metadata
I20260811 05:17:37.150180 6639 catalog_manager.cc:4841] TS fbeea6ac75b940f5bc9728abf4f50810 (127.7.67.130:46759): tablet d49b5f45d06f415da0529ce2f6befc5d (table test-table [id=e22d9c5a742c4516946306a7cc569f79]) successfully deleted
I20260811 05:17:37.150462 6639 catalog_manager.cc:4841] TS 393e7cf0ccab4eab95c16d5d51630051 (127.7.67.131:38975): tablet d49b5f45d06f415da0529ce2f6befc5d (table test-table [id=e22d9c5a742c4516946306a7cc569f79]) successfully deleted
W20260811 05:17:37.151850 6642 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.7.67.129:42315 (error 108)
I20260811 05:17:37.152765 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 6959
I20260811 05:17:37.153129 7373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:37.152751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.129:42315 (local address 127.0.0.1:44412)
0811 05:17:37.152934 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 05:17:37.153020 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.129:42315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 05:17:37.165696 6642 catalog_manager.cc:4584] TS 264006b018974602a619e40f953558f9 (127.7.67.129:42315): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d49b5f45d06f415da0529ce2f6befc5d: Network error: Client connection negotiation failed: client connection to 127.7.67.129:42315: connect: Connection refused (error 111)
I20260811 05:17:37.174782 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 7157
I20260811 05:17:37.200385 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 6614
2026-08-11T05:17:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1125 ms)
[----------] 6 tests from EncryptionPolicyTest (11851 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:37 dist-test-slave-04bb krb5kdc[7434](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:37 dist-test-slave-04bb krb5kdc[7434](info): set up 2 sockets
Aug 11 05:17:37 dist-test-slave-04bb krb5kdc[7434](info): commencing operation
krb5kdc: starting...
W20260811 05:17:38.475255 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:17:38 dist-test-slave-04bb krb5kdc[7434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:17:38.604368 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38191
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39263
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:38.717707 7460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:38.717999 7460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:38.718072 7460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:38.721997 7460 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:38.722082 7460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:38.722123 7460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:38.722160 7460 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:38.722203 7460 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:38.727303 7460 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39263
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38191
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:38.728343 7460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:38.729380 7460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 05:17:38.737888 7460 server_base.cc:1034] running on GCE node
W20260811 05:17:38.737854 7473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:38.738044 7467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:38.737846 7468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:38.738605 7460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:38.739751 7460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:38.740900 7460 hybrid_clock.cc:648] HybridClock initialized: now 1786425458740873 us; error 40 us; skew 500 ppm
Aug 11 05:17:38 dist-test-slave-04bb krb5kdc[7434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425458, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:38.746851 7460 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:17:38.747797 7460 webserver.cc:460] Webserver started at http://127.0.0.1:43727/ using document root <none> and password file <none>
I20260811 05:17:38.748373 7460 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:38.748461 7460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:38.748731 7460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:38.750849 7460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "82a9eb35976a445689df26f3c889c957"
format_stamp: "Formatted at 2026-08-11 05:17:38 on dist-test-slave-04bb"
server_key: "fb23760733dc218d122dde5ce912893a"
server_key_iv: "f2eb40a1c75c2d30218b14be93208fc7"
server_key_version: "encryptionkey@0"
I20260811 05:17:38.751407 7460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "82a9eb35976a445689df26f3c889c957"
format_stamp: "Formatted at 2026-08-11 05:17:38 on dist-test-slave-04bb"
server_key: "fb23760733dc218d122dde5ce912893a"
server_key_iv: "f2eb40a1c75c2d30218b14be93208fc7"
server_key_version: "encryptionkey@0"
I20260811 05:17:38.755160 7460 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 05:17:38.757597 7480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:38.758811 7460 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:17:38.758960 7460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "82a9eb35976a445689df26f3c889c957"
format_stamp: "Formatted at 2026-08-11 05:17:38 on dist-test-slave-04bb"
server_key: "fb23760733dc218d122dde5ce912893a"
server_key_iv: "f2eb40a1c75c2d30218b14be93208fc7"
server_key_version: "encryptionkey@0"
I20260811 05:17:38.759068 7460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:38.768136 7460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:38.770921 7460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:38.771107 7460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:38.778079 7460 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38191
I20260811 05:17:38.778106 7557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38191 every 8 connection(s)
I20260811 05:17:38.778991 7460 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:38.780767 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 7460
I20260811 05:17:38.780897 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:17:38.781168 7438 external_mini_cluster.cc:1428] Setting key d1095c2d19f60ba73807f476c338a310
I20260811 05:17:38.781818 7560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:38.787606 7560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82a9eb35976a445689df26f3c889c957: Bootstrap starting.
Aug 11 05:17:38 dist-test-slave-04bb krb5kdc[7434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425458, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 05:17:38.790200 7560 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82a9eb35976a445689df26f3c889c957: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:38.791091 7560 log.cc:824] T 00000000000000000000000000000000 P 82a9eb35976a445689df26f3c889c957: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:38.793304 7560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82a9eb35976a445689df26f3c889c957: No bootstrap required, opened a new log
I20260811 05:17:38.795625 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:38.782569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45957 (local address 127.0.0.1:38191)
0811 05:17:38.783027 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:38.783042 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:38.783079 (+ 37us) server_negotiation.cc:407] Connection header received
0811 05:17:38.783758 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:38.783775 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:38.784106 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:38.784453 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:38.785323 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:38.786312 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:38.787203 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:38.787674 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:38.787852 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:38.790784 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:38.790789 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:38.790802 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:38.790829 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:38.792624 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:38.793180 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:38.793184 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:38.793189 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:38.793318 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:38.793686 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:38.793689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:38.793690 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:38.794056 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:38.794241 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:38.794609 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 05:17:38.794893 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":152,"threads_started":1}
I20260811 05:17:38.797227 7560 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82a9eb35976a445689df26f3c889c957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82a9eb35976a445689df26f3c889c957" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38191 } }
I20260811 05:17:38.797497 7560 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82a9eb35976a445689df26f3c889c957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:38.797576 7560 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82a9eb35976a445689df26f3c889c957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82a9eb35976a445689df26f3c889c957, State: Initialized, Role: FOLLOWER
I20260811 05:17:38.798108 7560 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82a9eb35976a445689df26f3c889c957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82a9eb35976a445689df26f3c889c957" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38191 } }
I20260811 05:17:38.798266 7560 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82a9eb35976a445689df26f3c889c957 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:38.798349 7560 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82a9eb35976a445689df26f3c889c957 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:38.798456 7560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82a9eb35976a445689df26f3c889c957 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:38.798900 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:38.799546 7560 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82a9eb35976a445689df26f3c889c957 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82a9eb35976a445689df26f3c889c957" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38191 } }
I20260811 05:17:38.799949 7560 leader_election.cc:302] T 00000000000000000000000000000000 P 82a9eb35976a445689df26f3c889c957 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82a9eb35976a445689df26f3c889c957; no voters:
I20260811 05:17:38.800303 7560 leader_election.cc:288] T 00000000000000000000000000000000 P 82a9eb35976a445689df26f3c889c957 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:38.800511 7568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82a9eb35976a445689df26f3c889c957 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:38.800809 7568 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82a9eb35976a445689df26f3c889c957 [term 1 LEADER]: Becoming Leader. State: Replica: 82a9eb35976a445689df26f3c889c957, State: Running, Role: LEADER
I20260811 05:17:38.801275 7568 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82a9eb35976a445689df26f3c889c957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82a9eb35976a445689df26f3c889c957" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38191 } }
I20260811 05:17:38.801782 7560 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82a9eb35976a445689df26f3c889c957 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:38.803141 7575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82a9eb35976a445689df26f3c889c957 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82a9eb35976a445689df26f3c889c957" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82a9eb35976a445689df26f3c889c957" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38191 } } }
I20260811 05:17:38.803285 7575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82a9eb35976a445689df26f3c889c957 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:38.803689 7573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82a9eb35976a445689df26f3c889c957 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82a9eb35976a445689df26f3c889c957. Latest consensus state: current_term: 1 leader_uuid: "82a9eb35976a445689df26f3c889c957" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82a9eb35976a445689df26f3c889c957" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38191 } } }
I20260811 05:17:38.803843 7573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82a9eb35976a445689df26f3c889c957 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:38.804152 7589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:38.807842 7589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:38.810353 7564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:38.801313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39239 (local address 127.0.0.1:38191)
0811 05:17:38.801408 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:38.801413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:38.801629 (+ 216us) server_negotiation.cc:407] Connection header received
0811 05:17:38.801824 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:38.801827 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:38.801879 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:38.801957 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:38.802951 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:38.803784 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:38.804993 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:38.805247 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:38.805360 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:38.806787 (+ 1427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:38.806791 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:38.806794 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:38.806822 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:38.808293 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:38.808935 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:38.808939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:38.808946 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:38.809018 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:38.809478 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:38.809481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:38.809483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:38.809652 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:38.809821 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:38.810080 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 05:17:38.810172 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":26}
I20260811 05:17:38.813853 7589 catalog_manager.cc:1347] Generated new cluster ID: 559d121ad0c34b61afc07879a75da328
I20260811 05:17:38.813920 7589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:38.827075 7589 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:38.828231 7589 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:38.846232 7589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82a9eb35976a445689df26f3c889c957: Generated new TSK 0
I20260811 05:17:38.846935 7589 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:38.858567 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 7460
W20260811 05:17:38.875422 7428 connection.cc:535] client connection to 127.0.0.1:38191 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38191): Connection reset by peer (error 104)
2026-08-11T05:17:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1656 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:38 dist-test-slave-04bb krb5kdc[7605](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:38 dist-test-slave-04bb krb5kdc[7605](info): set up 2 sockets
Aug 11 05:17:38 dist-test-slave-04bb krb5kdc[7605](info): commencing operation
krb5kdc: starting...
W20260811 05:17:40.151834 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:17:40 dist-test-slave-04bb krb5kdc[7605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:17:40.283936 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46037
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46037
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:40.398567 7623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:40.398815 7623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:40.398856 7623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:40.402788 7623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:40.402840 7623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:40.402860 7623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:40.402876 7623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:40.402891 7623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:40.407970 7623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46037
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46037
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:40.408912 7623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:40.409858 7623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:40.418319 7630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:40.418308 7632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:40.418373 7623 server_base.cc:1034] running on GCE node
W20260811 05:17:40.418310 7634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:40.419039 7623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:40.420113 7623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:40.421295 7623 hybrid_clock.cc:648] HybridClock initialized: now 1786425460421284 us; error 30 us; skew 500 ppm
Aug 11 05:17:40 dist-test-slave-04bb krb5kdc[7605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425460, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:40.423985 7623 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:17:40.424883 7623 webserver.cc:460] Webserver started at http://127.0.0.1:35677/ using document root <none> and password file <none>
I20260811 05:17:40.425436 7623 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:40.425522 7623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:40.425750 7623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:40.427729 7623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "5cee1981975a4247bdaf3ab7c9551406"
format_stamp: "Formatted at 2026-08-11 05:17:40 on dist-test-slave-04bb"
server_key: "a8558f2b75208802e23860e76c5b43ac"
server_key_iv: "68b135e5d880042d6cc75935accd4eb6"
server_key_version: "encryptionkey@0"
I20260811 05:17:40.428305 7623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "5cee1981975a4247bdaf3ab7c9551406"
format_stamp: "Formatted at 2026-08-11 05:17:40 on dist-test-slave-04bb"
server_key: "a8558f2b75208802e23860e76c5b43ac"
server_key_iv: "68b135e5d880042d6cc75935accd4eb6"
server_key_version: "encryptionkey@0"
I20260811 05:17:40.432134 7623 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260811 05:17:40.434584 7645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:40.435716 7623 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 05:17:40.435859 7623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "5cee1981975a4247bdaf3ab7c9551406"
format_stamp: "Formatted at 2026-08-11 05:17:40 on dist-test-slave-04bb"
server_key: "a8558f2b75208802e23860e76c5b43ac"
server_key_iv: "68b135e5d880042d6cc75935accd4eb6"
server_key_version: "encryptionkey@0"
I20260811 05:17:40.435967 7623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:40.457751 7623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:40.460768 7623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:40.460990 7623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:40.468482 7623 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46037
I20260811 05:17:40.468513 7729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46037 every 8 connection(s)
I20260811 05:17:40.469434 7623 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:40.470170 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 7623
I20260811 05:17:40.470314 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:17:40.470587 7438 external_mini_cluster.cc:1428] Setting key 827fa5015f0aa228c8124acd46716986
I20260811 05:17:40.472682 7733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:40.479583 7733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cee1981975a4247bdaf3ab7c9551406: Bootstrap starting.
I20260811 05:17:40.480635 7737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:40.471999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58957 (local address 127.0.0.1:46037)
0811 05:17:40.472434 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:40.472451 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:40.472492 (+ 41us) server_negotiation.cc:407] Connection header received
0811 05:17:40.473387 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:40.473410 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:40.473873 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:40.474234 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:40.474633 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:40.476126 (+ 1493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:40.477051 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:40.477733 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:40.477885 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:40.478730 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:40.478736 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:40.478751 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:40.479241 (+ 490us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:40.479305 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:40.479334 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:40.479468 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:40.479615 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 05:17:40.479757 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":124,"threads_started":1}
I20260811 05:17:40.482275 7733 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5cee1981975a4247bdaf3ab7c9551406: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:40.483180 7733 log.cc:824] T 00000000000000000000000000000000 P 5cee1981975a4247bdaf3ab7c9551406: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:40.483560 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:40.485198 7733 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5cee1981975a4247bdaf3ab7c9551406: No bootstrap required, opened a new log
I20260811 05:17:40.489554 7737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:40.485480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57967 (local address 127.0.0.1:46037)
0811 05:17:40.485599 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:40.485603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:40.485805 (+ 202us) server_negotiation.cc:407] Connection header received
0811 05:17:40.485927 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:40.485929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:40.485962 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:40.486022 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:40.486312 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:40.486789 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:40.487991 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:40.488182 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:40.488254 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:40.489097 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:40.489100 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:40.489102 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:40.489113 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:40.489136 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:40.489146 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:40.489208 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:40.489359 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 05:17:40.489388 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 05:17:40.489817 7733 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5cee1981975a4247bdaf3ab7c9551406 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cee1981975a4247bdaf3ab7c9551406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46037 } }
I20260811 05:17:40.490114 7733 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5cee1981975a4247bdaf3ab7c9551406 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:40.490185 7733 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5cee1981975a4247bdaf3ab7c9551406 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cee1981975a4247bdaf3ab7c9551406, State: Initialized, Role: FOLLOWER
I20260811 05:17:40.490684 7733 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5cee1981975a4247bdaf3ab7c9551406 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cee1981975a4247bdaf3ab7c9551406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46037 } }
I20260811 05:17:40.490849 7733 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5cee1981975a4247bdaf3ab7c9551406 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:40.490902 7733 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5cee1981975a4247bdaf3ab7c9551406 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:40.490975 7733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5cee1981975a4247bdaf3ab7c9551406 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:40.491793 7733 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5cee1981975a4247bdaf3ab7c9551406 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cee1981975a4247bdaf3ab7c9551406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46037 } }
I20260811 05:17:40.492085 7733 leader_election.cc:302] T 00000000000000000000000000000000 P 5cee1981975a4247bdaf3ab7c9551406 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5cee1981975a4247bdaf3ab7c9551406; no voters:
I20260811 05:17:40.492338 7733 leader_election.cc:288] T 00000000000000000000000000000000 P 5cee1981975a4247bdaf3ab7c9551406 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:40.492463 7746 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5cee1981975a4247bdaf3ab7c9551406 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:40.492730 7746 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5cee1981975a4247bdaf3ab7c9551406 [term 1 LEADER]: Becoming Leader. State: Replica: 5cee1981975a4247bdaf3ab7c9551406, State: Running, Role: LEADER
I20260811 05:17:40.493077 7746 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5cee1981975a4247bdaf3ab7c9551406 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cee1981975a4247bdaf3ab7c9551406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46037 } }
I20260811 05:17:40.493571 7733 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5cee1981975a4247bdaf3ab7c9551406 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:40.494837 7748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cee1981975a4247bdaf3ab7c9551406 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5cee1981975a4247bdaf3ab7c9551406" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cee1981975a4247bdaf3ab7c9551406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46037 } } }
I20260811 05:17:40.494851 7750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5cee1981975a4247bdaf3ab7c9551406 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5cee1981975a4247bdaf3ab7c9551406. Latest consensus state: current_term: 1 leader_uuid: "5cee1981975a4247bdaf3ab7c9551406" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5cee1981975a4247bdaf3ab7c9551406" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46037 } } }
I20260811 05:17:40.494987 7748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cee1981975a4247bdaf3ab7c9551406 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:40.495083 7750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5cee1981975a4247bdaf3ab7c9551406 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:40.495512 7765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:40.498952 7765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:40.504568 7765 catalog_manager.cc:1347] Generated new cluster ID: 6b9c5414ccea47288621ce3aeb2bd944
I20260811 05:17:40.504633 7765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:40.514001 7765 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:40.515190 7765 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:40.525358 7765 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5cee1981975a4247bdaf3ab7c9551406: Generated new TSK 0
I20260811 05:17:40.526096 7765 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:40.538303 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 7623
W20260811 05:17:40.554311 7601 connection.cc:535] client connection to 127.0.0.1:46037 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46037): Connection reset by peer (error 104)
2026-08-11T05:17:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1678 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:40 dist-test-slave-04bb krb5kdc[7779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:40 dist-test-slave-04bb krb5kdc[7779](info): set up 2 sockets
Aug 11 05:17:40 dist-test-slave-04bb krb5kdc[7779](info): commencing operation
krb5kdc: starting...
W20260811 05:17:41.816756 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:17:41 dist-test-slave-04bb krb5kdc[7779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425461, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:17:41.947649 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40887
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41067
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:42.062122 7799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:42.062413 7799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:42.062491 7799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:42.066468 7799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:42.066550 7799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:42.066586 7799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:42.066639 7799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:42.066679 7799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:42.071939 7799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41067
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40887
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:42.073127 7799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:42.074179 7799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:42.082866 7812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:42.082893 7806 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:42.082890 7807 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:42.082937 7799 server_base.cc:1034] running on GCE node
I20260811 05:17:42.083691 7799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:42.084856 7799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:42.086019 7799 hybrid_clock.cc:648] HybridClock initialized: now 1786425462086001 us; error 33 us; skew 500 ppm
Aug 11 05:17:42 dist-test-slave-04bb krb5kdc[7779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425462, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:42.089033 7799 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:17:42.090022 7799 webserver.cc:460] Webserver started at http://127.0.0.1:40143/ using document root <none> and password file <none>
I20260811 05:17:42.090603 7799 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:42.090692 7799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:42.090927 7799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:42.093659 7799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "5dc30afc4b264e089148b070111a6119"
format_stamp: "Formatted at 2026-08-11 05:17:42 on dist-test-slave-04bb"
server_key: "b3593adc99b33b7d051e093f967f90d1"
server_key_iv: "4d02439fa3288aa346d460bba8812cf5"
server_key_version: "encryptionkey@0"
I20260811 05:17:42.094238 7799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "5dc30afc4b264e089148b070111a6119"
format_stamp: "Formatted at 2026-08-11 05:17:42 on dist-test-slave-04bb"
server_key: "b3593adc99b33b7d051e093f967f90d1"
server_key_iv: "4d02439fa3288aa346d460bba8812cf5"
server_key_version: "encryptionkey@0"
I20260811 05:17:42.099613 7799 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 05:17:42.102699 7821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:42.104007 7799 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 05:17:42.104156 7799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "5dc30afc4b264e089148b070111a6119"
format_stamp: "Formatted at 2026-08-11 05:17:42 on dist-test-slave-04bb"
server_key: "b3593adc99b33b7d051e093f967f90d1"
server_key_iv: "4d02439fa3288aa346d460bba8812cf5"
server_key_version: "encryptionkey@0"
I20260811 05:17:42.104274 7799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:42.109009 7799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:42.112016 7799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:42.112206 7799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:42.119536 7799 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40887
I20260811 05:17:42.119549 7899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40887 every 8 connection(s)
I20260811 05:17:42.120452 7799 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:42.123343 7900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:42.123932 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 7799
I20260811 05:17:42.124089 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:17:42.124377 7438 external_mini_cluster.cc:1428] Setting key 997310f6b39911572f342315bc55bafb
I20260811 05:17:42.129473 7904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:42.125695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46951 (local address 127.0.0.1:40887)
0811 05:17:42.126107 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:42.126121 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:42.126157 (+ 36us) server_negotiation.cc:407] Connection header received
0811 05:17:42.126784 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:42.126800 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:42.127075 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:42.127350 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:42.128218 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:42.128223 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:42.128236 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:42.128273 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:42.128338 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:42.128362 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:42.128473 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:42.128625 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 05:17:42.128778 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":122,"threads_started":1}
I20260811 05:17:42.129659 7900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dc30afc4b264e089148b070111a6119: Bootstrap starting.
I20260811 05:17:42.132169 7900 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5dc30afc4b264e089148b070111a6119: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:42.132304 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:42.132992 7900 log.cc:824] T 00000000000000000000000000000000 P 5dc30afc4b264e089148b070111a6119: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:42.135079 7900 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dc30afc4b264e089148b070111a6119: No bootstrap required, opened a new log
I20260811 05:17:42.136021 7904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:42.134364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41091 (local address 127.0.0.1:40887)
0811 05:17:42.134470 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:42.134473 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:42.134693 (+ 220us) server_negotiation.cc:407] Connection header received
0811 05:17:42.134840 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:42.134843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:42.134878 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:42.134952 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:42.135645 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:42.135648 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:42.135651 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:42.135662 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:42.135688 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:42.135704 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:42.135788 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:42.135872 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 05:17:42.135910 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 05:17:42.138988 7900 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5dc30afc4b264e089148b070111a6119 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc30afc4b264e089148b070111a6119" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40887 } }
I20260811 05:17:42.139209 7900 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5dc30afc4b264e089148b070111a6119 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:42.139295 7900 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5dc30afc4b264e089148b070111a6119 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dc30afc4b264e089148b070111a6119, State: Initialized, Role: FOLLOWER
I20260811 05:17:42.139734 7900 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5dc30afc4b264e089148b070111a6119 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc30afc4b264e089148b070111a6119" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40887 } }
I20260811 05:17:42.139878 7900 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5dc30afc4b264e089148b070111a6119 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:42.139959 7900 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5dc30afc4b264e089148b070111a6119 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:42.140075 7900 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5dc30afc4b264e089148b070111a6119 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:42.141012 7900 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5dc30afc4b264e089148b070111a6119 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc30afc4b264e089148b070111a6119" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40887 } }
I20260811 05:17:42.141352 7900 leader_election.cc:302] T 00000000000000000000000000000000 P 5dc30afc4b264e089148b070111a6119 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5dc30afc4b264e089148b070111a6119; no voters:
I20260811 05:17:42.141644 7900 leader_election.cc:288] T 00000000000000000000000000000000 P 5dc30afc4b264e089148b070111a6119 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:42.141800 7922 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5dc30afc4b264e089148b070111a6119 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:42.142027 7922 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5dc30afc4b264e089148b070111a6119 [term 1 LEADER]: Becoming Leader. State: Replica: 5dc30afc4b264e089148b070111a6119, State: Running, Role: LEADER
I20260811 05:17:42.142357 7922 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5dc30afc4b264e089148b070111a6119 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc30afc4b264e089148b070111a6119" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40887 } }
I20260811 05:17:42.142853 7900 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5dc30afc4b264e089148b070111a6119 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:42.143990 7926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dc30afc4b264e089148b070111a6119 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5dc30afc4b264e089148b070111a6119. Latest consensus state: current_term: 1 leader_uuid: "5dc30afc4b264e089148b070111a6119" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc30afc4b264e089148b070111a6119" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40887 } } }
I20260811 05:17:42.144130 7926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dc30afc4b264e089148b070111a6119 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:42.144034 7925 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dc30afc4b264e089148b070111a6119 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5dc30afc4b264e089148b070111a6119" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dc30afc4b264e089148b070111a6119" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40887 } } }
I20260811 05:17:42.144397 7925 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dc30afc4b264e089148b070111a6119 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:42.144663 7938 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:42.148300 7938 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:42.153664 7938 catalog_manager.cc:1347] Generated new cluster ID: 7eea2cdf3bdc4bf58171334c97391644
I20260811 05:17:42.153729 7938 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:42.173635 7938 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:42.174034 7938 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:42.186902 7938 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5dc30afc4b264e089148b070111a6119: Generated new TSK 0
I20260811 05:17:42.187716 7938 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:42.191210 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 7799
2026-08-11T05:17:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1651 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:42 dist-test-slave-04bb krb5kdc[7965](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:42 dist-test-slave-04bb krb5kdc[7965](info): set up 2 sockets
Aug 11 05:17:42 dist-test-slave-04bb krb5kdc[7965](info): commencing operation
krb5kdc: starting...
W20260811 05:17:43.466001 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:17:43 dist-test-slave-04bb krb5kdc[7965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:17:43.597744 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35743
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35743
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:43.714267 7982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:43.714516 7982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:43.714557 7982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:43.718918 7982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:43.718971 7982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:43.718991 7982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:43.719007 7982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:43.719023 7982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:43.724201 7982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35743
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35743
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:43.725207 7982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:43.726151 7982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:43.734541 7988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:43.734602 7982 server_base.cc:1034] running on GCE node
W20260811 05:17:43.734508 7990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:43.734529 7987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:43.735350 7982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:43.736428 7982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:43.737607 7982 hybrid_clock.cc:648] HybridClock initialized: now 1786425463737591 us; error 28 us; skew 500 ppm
Aug 11 05:17:43 dist-test-slave-04bb krb5kdc[7965](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425463, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:43.740520 7982 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:17:43.741673 7982 webserver.cc:460] Webserver started at http://127.0.0.1:35991/ using document root <none> and password file <none>
I20260811 05:17:43.742252 7982 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:43.742345 7982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:43.742589 7982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:43.744721 7982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "bd9e8fd9407f4f4ca974c64d3be22bf7"
format_stamp: "Formatted at 2026-08-11 05:17:43 on dist-test-slave-04bb"
server_key: "5140e2c308f58beee609a699a2e9aedf"
server_key_iv: "5adc6c4c0635e22cb00465bb73d61afb"
server_key_version: "encryptionkey@0"
I20260811 05:17:43.745306 7982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "bd9e8fd9407f4f4ca974c64d3be22bf7"
format_stamp: "Formatted at 2026-08-11 05:17:43 on dist-test-slave-04bb"
server_key: "5140e2c308f58beee609a699a2e9aedf"
server_key_iv: "5adc6c4c0635e22cb00465bb73d61afb"
server_key_version: "encryptionkey@0"
I20260811 05:17:43.749056 7982 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:17:43.751533 8000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:43.753147 7982 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:17:43.753299 7982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "bd9e8fd9407f4f4ca974c64d3be22bf7"
format_stamp: "Formatted at 2026-08-11 05:17:43 on dist-test-slave-04bb"
server_key: "5140e2c308f58beee609a699a2e9aedf"
server_key_iv: "5adc6c4c0635e22cb00465bb73d61afb"
server_key_version: "encryptionkey@0"
I20260811 05:17:43.753414 7982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:43.779983 7982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:43.783203 7982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:43.783422 7982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:43.790876 7982 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35743
I20260811 05:17:43.790907 8072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35743 every 8 connection(s)
I20260811 05:17:43.791854 7982 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:43.794732 8073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:43.795166 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 7982
I20260811 05:17:43.795337 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:17:43.795648 7438 external_mini_cluster.cc:1428] Setting key 7b6ac8e922dfa1c4cc238cb388c384f5
I20260811 05:17:43.801654 8073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd9e8fd9407f4f4ca974c64d3be22bf7: Bootstrap starting.
I20260811 05:17:43.804226 8073 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd9e8fd9407f4f4ca974c64d3be22bf7: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:43.805249 8073 log.cc:824] T 00000000000000000000000000000000 P bd9e8fd9407f4f4ca974c64d3be22bf7: Log is configured to *not* fsync() on all Append() calls
Aug 11 05:17:43 dist-test-slave-04bb krb5kdc[7965](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425463, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 05:17:43.807757 8073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd9e8fd9407f4f4ca974c64d3be22bf7: No bootstrap required, opened a new log
I20260811 05:17:43.812706 8076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:43.797271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52199 (local address 127.0.0.1:35743)
0811 05:17:43.797708 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:43.797736 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:43.797781 (+ 45us) server_negotiation.cc:407] Connection header received
0811 05:17:43.798631 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:43.798656 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:43.799091 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:43.799473 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:43.800239 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:43.801794 (+ 1555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:43.802791 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:43.803481 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:43.803646 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:43.806998 (+ 3352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:43.807004 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:43.807020 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:43.807054 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:43.809236 (+ 2182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:43.809741 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:43.809748 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:43.809755 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:43.809915 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:43.810236 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:43.810242 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:43.810246 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:43.810831 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:43.811043 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:43.811325 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 05:17:43.811644 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":115,"threads_started":1}
I20260811 05:17:43.812706 8073 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd9e8fd9407f4f4ca974c64d3be22bf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd9e8fd9407f4f4ca974c64d3be22bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35743 } }
I20260811 05:17:43.813015 8073 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd9e8fd9407f4f4ca974c64d3be22bf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:43.813077 8073 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd9e8fd9407f4f4ca974c64d3be22bf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd9e8fd9407f4f4ca974c64d3be22bf7, State: Initialized, Role: FOLLOWER
I20260811 05:17:43.813544 8073 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd9e8fd9407f4f4ca974c64d3be22bf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd9e8fd9407f4f4ca974c64d3be22bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35743 } }
I20260811 05:17:43.813714 8073 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd9e8fd9407f4f4ca974c64d3be22bf7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:43.814504 8073 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd9e8fd9407f4f4ca974c64d3be22bf7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:43.814647 8073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd9e8fd9407f4f4ca974c64d3be22bf7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:43.815747 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:43.815860 8073 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd9e8fd9407f4f4ca974c64d3be22bf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd9e8fd9407f4f4ca974c64d3be22bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35743 } }
I20260811 05:17:43.816222 8073 leader_election.cc:302] T 00000000000000000000000000000000 P bd9e8fd9407f4f4ca974c64d3be22bf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd9e8fd9407f4f4ca974c64d3be22bf7; no voters:
I20260811 05:17:43.816591 8073 leader_election.cc:288] T 00000000000000000000000000000000 P bd9e8fd9407f4f4ca974c64d3be22bf7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:43.816685 8079 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd9e8fd9407f4f4ca974c64d3be22bf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:43.816915 8079 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd9e8fd9407f4f4ca974c64d3be22bf7 [term 1 LEADER]: Becoming Leader. State: Replica: bd9e8fd9407f4f4ca974c64d3be22bf7, State: Running, Role: LEADER
I20260811 05:17:43.817265 8079 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd9e8fd9407f4f4ca974c64d3be22bf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd9e8fd9407f4f4ca974c64d3be22bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35743 } }
I20260811 05:17:43.818012 8073 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd9e8fd9407f4f4ca974c64d3be22bf7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:43.818529 8086 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd9e8fd9407f4f4ca974c64d3be22bf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd9e8fd9407f4f4ca974c64d3be22bf7. Latest consensus state: current_term: 1 leader_uuid: "bd9e8fd9407f4f4ca974c64d3be22bf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd9e8fd9407f4f4ca974c64d3be22bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35743 } } }
I20260811 05:17:43.818671 8086 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd9e8fd9407f4f4ca974c64d3be22bf7 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:43.819023 8081 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd9e8fd9407f4f4ca974c64d3be22bf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd9e8fd9407f4f4ca974c64d3be22bf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd9e8fd9407f4f4ca974c64d3be22bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35743 } } }
I20260811 05:17:43.819113 8081 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd9e8fd9407f4f4ca974c64d3be22bf7 [sys.catalog]: This master's current role is: LEADER
W20260811 05:17:43.823757 8115 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bd9e8fd9407f4f4ca974c64d3be22bf7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:17:43.823845 8115 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:17:43.823925 8101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:43.824685 8101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:43.829569 8076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:43.818196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60381 (local address 127.0.0.1:35743)
0811 05:17:43.818300 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:43.818305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:43.819454 (+ 1149us) server_negotiation.cc:407] Connection header received
0811 05:17:43.819586 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:43.819590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:43.819648 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:43.819738 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:43.820786 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:43.821429 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:43.822532 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:43.822862 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:43.822970 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:43.825643 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:43.825647 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:43.825650 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:43.825676 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:43.827497 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:43.828036 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:43.828039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:43.828041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:43.828134 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:43.828471 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:43.828475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:43.828477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:43.828736 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:43.828846 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:43.829238 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 05:17:43.829369 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 05:17:43.831266 8101 catalog_manager.cc:1347] Generated new cluster ID: fd0328593dfc45a685a626eb5429ade7
I20260811 05:17:43.831347 8101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:43.854802 8101 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:43.855792 8101 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:43.865520 8101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd9e8fd9407f4f4ca974c64d3be22bf7: Generated new TSK 0
I20260811 05:17:43.866372 8101 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:43.879007 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 7982
2026-08-11T05:17:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1684 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:43 dist-test-slave-04bb krb5kdc[8124](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:43 dist-test-slave-04bb krb5kdc[8124](info): set up 2 sockets
Aug 11 05:17:43 dist-test-slave-04bb krb5kdc[8124](info): commencing operation
krb5kdc: starting...
W20260811 05:17:45.146530 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:17:45 dist-test-slave-04bb krb5kdc[8124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425465, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:17:45.276476 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41875
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:45.395850 8147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:45.396111 8147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:45.396153 8147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:45.400166 8147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:45.400228 8147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:45.400247 8147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:45.400264 8147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:45.400338 8147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:45.405603 8147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41875
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:45.406706 8147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:45.407770 8147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:45.416639 8160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:45.416648 8157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:45.416639 8163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:45.416854 8147 server_base.cc:1034] running on GCE node
I20260811 05:17:45.417460 8147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:45.418655 8147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:45.419852 8147 hybrid_clock.cc:648] HybridClock initialized: now 1786425465419839 us; error 26 us; skew 500 ppm
Aug 11 05:17:45 dist-test-slave-04bb krb5kdc[8124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425465, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:45.422825 8147 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:17:45.423751 8147 webserver.cc:460] Webserver started at http://127.0.0.1:40399/ using document root <none> and password file <none>
I20260811 05:17:45.424289 8147 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:45.424347 8147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:45.424515 8147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:45.426621 8147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "d88f22522cc8432ba841869b18d2e9df"
format_stamp: "Formatted at 2026-08-11 05:17:45 on dist-test-slave-04bb"
server_key: "e0f972efcf517e79da2d18a8139cacb7"
server_key_iv: "20e414932f63ae5991b58f5c08cf5804"
server_key_version: "encryptionkey@0"
I20260811 05:17:45.427129 8147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "d88f22522cc8432ba841869b18d2e9df"
format_stamp: "Formatted at 2026-08-11 05:17:45 on dist-test-slave-04bb"
server_key: "e0f972efcf517e79da2d18a8139cacb7"
server_key_iv: "20e414932f63ae5991b58f5c08cf5804"
server_key_version: "encryptionkey@0"
I20260811 05:17:45.430893 8147 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 05:17:45.433394 8173 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:45.434692 8147 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:17:45.434821 8147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "d88f22522cc8432ba841869b18d2e9df"
format_stamp: "Formatted at 2026-08-11 05:17:45 on dist-test-slave-04bb"
server_key: "e0f972efcf517e79da2d18a8139cacb7"
server_key_iv: "20e414932f63ae5991b58f5c08cf5804"
server_key_version: "encryptionkey@0"
I20260811 05:17:45.434930 8147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:45.451565 8147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:45.454903 8147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:45.455104 8147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:45.462728 8147 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41875
I20260811 05:17:45.462751 8251 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41875 every 8 connection(s)
I20260811 05:17:45.463794 8147 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:45.466774 8252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:45.472620 8252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d88f22522cc8432ba841869b18d2e9df: Bootstrap starting.
I20260811 05:17:45.473426 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 8147
I20260811 05:17:45.473570 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:17:45.473846 7438 external_mini_cluster.cc:1428] Setting key cad358c5e57b5453f007328239b6869d
I20260811 05:17:45.475538 8252 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d88f22522cc8432ba841869b18d2e9df: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:45.476310 8252 log.cc:824] T 00000000000000000000000000000000 P d88f22522cc8432ba841869b18d2e9df: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:45.478286 8252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d88f22522cc8432ba841869b18d2e9df: No bootstrap required, opened a new log
I20260811 05:17:45.481986 8252 raft_consensus.cc:348] T 00000000000000000000000000000000 P d88f22522cc8432ba841869b18d2e9df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d88f22522cc8432ba841869b18d2e9df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41875 } }
I20260811 05:17:45.482333 8252 raft_consensus.cc:374] T 00000000000000000000000000000000 P d88f22522cc8432ba841869b18d2e9df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:45.482416 8252 raft_consensus.cc:729] T 00000000000000000000000000000000 P d88f22522cc8432ba841869b18d2e9df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d88f22522cc8432ba841869b18d2e9df, State: Initialized, Role: FOLLOWER
I20260811 05:17:45.482913 8252 consensus_queue.cc:260] T 00000000000000000000000000000000 P d88f22522cc8432ba841869b18d2e9df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d88f22522cc8432ba841869b18d2e9df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41875 } }
I20260811 05:17:45.482988 8256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:45.474993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54837 (local address 127.0.0.1:41875)
0811 05:17:45.475488 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:45.475503 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:45.475542 (+ 39us) server_negotiation.cc:407] Connection header received
0811 05:17:45.476279 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:45.476299 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:45.476765 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:45.477116 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:45.477589 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:45.478615 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:45.479528 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:45.480014 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:45.480184 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:45.481169 (+ 985us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:45.481176 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:45.481190 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:45.481576 (+ 386us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:45.481628 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:45.481661 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:45.481772 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:45.481921 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 05:17:45.482130 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":144,"threads_started":1}
I20260811 05:17:45.483088 8252 raft_consensus.cc:388] T 00000000000000000000000000000000 P d88f22522cc8432ba841869b18d2e9df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:45.483170 8252 raft_consensus.cc:482] T 00000000000000000000000000000000 P d88f22522cc8432ba841869b18d2e9df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:45.483312 8252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d88f22522cc8432ba841869b18d2e9df [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:45.484422 8252 raft_consensus.cc:504] T 00000000000000000000000000000000 P d88f22522cc8432ba841869b18d2e9df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d88f22522cc8432ba841869b18d2e9df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41875 } }
I20260811 05:17:45.484798 8252 leader_election.cc:302] T 00000000000000000000000000000000 P d88f22522cc8432ba841869b18d2e9df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d88f22522cc8432ba841869b18d2e9df; no voters:
I20260811 05:17:45.485157 8252 leader_election.cc:288] T 00000000000000000000000000000000 P d88f22522cc8432ba841869b18d2e9df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:45.485249 8258 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d88f22522cc8432ba841869b18d2e9df [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:45.485759 8258 raft_consensus.cc:686] T 00000000000000000000000000000000 P d88f22522cc8432ba841869b18d2e9df [term 1 LEADER]: Becoming Leader. State: Replica: d88f22522cc8432ba841869b18d2e9df, State: Running, Role: LEADER
I20260811 05:17:45.486184 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:45.486151 8258 consensus_queue.cc:237] T 00000000000000000000000000000000 P d88f22522cc8432ba841869b18d2e9df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d88f22522cc8432ba841869b18d2e9df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41875 } }
I20260811 05:17:45.486550 8252 sys_catalog.cc:564] T 00000000000000000000000000000000 P d88f22522cc8432ba841869b18d2e9df [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:45.487908 8262 sys_catalog.cc:455] T 00000000000000000000000000000000 P d88f22522cc8432ba841869b18d2e9df [sys.catalog]: SysCatalogTable state changed. Reason: New leader d88f22522cc8432ba841869b18d2e9df. Latest consensus state: current_term: 1 leader_uuid: "d88f22522cc8432ba841869b18d2e9df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d88f22522cc8432ba841869b18d2e9df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41875 } } }
I20260811 05:17:45.488041 8262 sys_catalog.cc:458] T 00000000000000000000000000000000 P d88f22522cc8432ba841869b18d2e9df [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:45.488432 8260 sys_catalog.cc:455] T 00000000000000000000000000000000 P d88f22522cc8432ba841869b18d2e9df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d88f22522cc8432ba841869b18d2e9df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d88f22522cc8432ba841869b18d2e9df" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41875 } } }
I20260811 05:17:45.488554 8260 sys_catalog.cc:458] T 00000000000000000000000000000000 P d88f22522cc8432ba841869b18d2e9df [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:45.488763 8276 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:45.492108 8276 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:45.493594 8256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:45.488595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49799 (local address 127.0.0.1:41875)
0811 05:17:45.488711 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:45.488715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:45.489240 (+ 525us) server_negotiation.cc:407] Connection header received
0811 05:17:45.489414 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:45.489420 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:45.489469 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:45.489552 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:45.489839 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:45.490377 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:45.491581 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:45.491813 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:45.491923 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:45.493178 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:45.493183 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:45.493185 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:45.493199 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:45.493233 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:45.493245 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:45.493326 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:45.493416 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 05:17:45.493465 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 05:17:45.498052 8276 catalog_manager.cc:1347] Generated new cluster ID: 08a31a2879f247c6a60e2cdffb93a317
I20260811 05:17:45.498121 8276 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:45.530007 8276 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:45.531360 8276 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:45.547616 8276 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d88f22522cc8432ba841869b18d2e9df: Generated new TSK 0
I20260811 05:17:45.548635 8276 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:45.555588 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 8147
2026-08-11T05:17:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1685 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:45 dist-test-slave-04bb krb5kdc[8300](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:45 dist-test-slave-04bb krb5kdc[8300](info): set up 2 sockets
Aug 11 05:17:45 dist-test-slave-04bb krb5kdc[8300](info): commencing operation
krb5kdc: starting...
W20260811 05:17:46.848387 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:17:46 dist-test-slave-04bb krb5kdc[8300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425466, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:17:46.981384 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38001
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38001
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:47.099180 8318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:47.099591 8318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:47.099694 8318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:47.106889 8318 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:47.107033 8318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:47.107084 8318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:47.107116 8318 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:47.107193 8318 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:47.116740 8318 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38001
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38001
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:47.119649 8318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:47.121090 8318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:47.130107 8334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:47.130309 8318 server_base.cc:1034] running on GCE node
W20260811 05:17:47.130167 8326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:47.130119 8325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:47.130975 8318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:47.132046 8318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:47.133237 8318 hybrid_clock.cc:648] HybridClock initialized: now 1786425467133213 us; error 46 us; skew 500 ppm
Aug 11 05:17:47 dist-test-slave-04bb krb5kdc[8300](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425467, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:47.136047 8318 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:17:47.137045 8318 webserver.cc:460] Webserver started at http://127.0.0.1:38737/ using document root <none> and password file <none>
I20260811 05:17:47.137621 8318 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:47.137725 8318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:47.137984 8318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:47.140053 8318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "36274b6d931c4f47975a3dd9c329e22d"
format_stamp: "Formatted at 2026-08-11 05:17:47 on dist-test-slave-04bb"
server_key: "93777af58d71182be1e0793f3b23f91e"
server_key_iv: "67fd495f8837cc80c7600947faebad85"
server_key_version: "encryptionkey@0"
I20260811 05:17:47.140699 8318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "36274b6d931c4f47975a3dd9c329e22d"
format_stamp: "Formatted at 2026-08-11 05:17:47 on dist-test-slave-04bb"
server_key: "93777af58d71182be1e0793f3b23f91e"
server_key_iv: "67fd495f8837cc80c7600947faebad85"
server_key_version: "encryptionkey@0"
I20260811 05:17:47.144414 8318 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 05:17:47.146855 8345 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:47.148142 8318 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:17:47.148316 8318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "36274b6d931c4f47975a3dd9c329e22d"
format_stamp: "Formatted at 2026-08-11 05:17:47 on dist-test-slave-04bb"
server_key: "93777af58d71182be1e0793f3b23f91e"
server_key_iv: "67fd495f8837cc80c7600947faebad85"
server_key_version: "encryptionkey@0"
I20260811 05:17:47.148437 8318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:47.153172 8318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:47.155979 8318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:47.156175 8318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:47.163434 8318 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38001
I20260811 05:17:47.163444 8421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38001 every 8 connection(s)
I20260811 05:17:47.164363 8318 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:47.167279 8423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:47.168119 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 8318
I20260811 05:17:47.168280 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:17:47.168625 7438 external_mini_cluster.cc:1428] Setting key b95d50dfa75b3201cbca53151109d334
I20260811 05:17:47.173916 8423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36274b6d931c4f47975a3dd9c329e22d: Bootstrap starting.
I20260811 05:17:47.173939 8426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:47.170062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41747 (local address 127.0.0.1:38001)
0811 05:17:47.170520 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:47.170534 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:47.170570 (+ 36us) server_negotiation.cc:407] Connection header received
0811 05:17:47.171220 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:47.171238 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:47.171523 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:47.171878 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:47.172762 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:47.172772 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:47.172785 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:47.172796 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:47.172854 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:47.172873 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:47.173027 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:47.173101 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 05:17:47.173267 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":147,"threads_started":1}
I20260811 05:17:47.176528 8423 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 36274b6d931c4f47975a3dd9c329e22d: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:47.176944 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:47.177380 8423 log.cc:824] T 00000000000000000000000000000000 P 36274b6d931c4f47975a3dd9c329e22d: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:47.179514 8423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 36274b6d931c4f47975a3dd9c329e22d: No bootstrap required, opened a new log
I20260811 05:17:47.180892 8426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:47.179234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35671 (local address 127.0.0.1:38001)
0811 05:17:47.179362 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:47.179366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:47.179522 (+ 156us) server_negotiation.cc:407] Connection header received
0811 05:17:47.179685 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:47.179688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:47.179721 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:47.179795 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:47.180496 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:47.180499 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:47.180502 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:47.180513 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:47.180569 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:47.180584 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:47.180680 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:47.180733 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 05:17:47.180770 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 05:17:47.183552 8423 raft_consensus.cc:348] T 00000000000000000000000000000000 P 36274b6d931c4f47975a3dd9c329e22d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36274b6d931c4f47975a3dd9c329e22d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38001 } }
I20260811 05:17:47.183768 8423 raft_consensus.cc:374] T 00000000000000000000000000000000 P 36274b6d931c4f47975a3dd9c329e22d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:47.183811 8423 raft_consensus.cc:729] T 00000000000000000000000000000000 P 36274b6d931c4f47975a3dd9c329e22d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36274b6d931c4f47975a3dd9c329e22d, State: Initialized, Role: FOLLOWER
I20260811 05:17:47.184166 8423 consensus_queue.cc:260] T 00000000000000000000000000000000 P 36274b6d931c4f47975a3dd9c329e22d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36274b6d931c4f47975a3dd9c329e22d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38001 } }
I20260811 05:17:47.184276 8423 raft_consensus.cc:388] T 00000000000000000000000000000000 P 36274b6d931c4f47975a3dd9c329e22d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:47.184315 8423 raft_consensus.cc:482] T 00000000000000000000000000000000 P 36274b6d931c4f47975a3dd9c329e22d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:47.184384 8423 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 36274b6d931c4f47975a3dd9c329e22d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:47.185242 8423 raft_consensus.cc:504] T 00000000000000000000000000000000 P 36274b6d931c4f47975a3dd9c329e22d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36274b6d931c4f47975a3dd9c329e22d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38001 } }
I20260811 05:17:47.185547 8423 leader_election.cc:302] T 00000000000000000000000000000000 P 36274b6d931c4f47975a3dd9c329e22d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 36274b6d931c4f47975a3dd9c329e22d; no voters:
I20260811 05:17:47.185801 8423 leader_election.cc:288] T 00000000000000000000000000000000 P 36274b6d931c4f47975a3dd9c329e22d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:47.185933 8434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 36274b6d931c4f47975a3dd9c329e22d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:47.186198 8434 raft_consensus.cc:686] T 00000000000000000000000000000000 P 36274b6d931c4f47975a3dd9c329e22d [term 1 LEADER]: Becoming Leader. State: Replica: 36274b6d931c4f47975a3dd9c329e22d, State: Running, Role: LEADER
I20260811 05:17:47.186573 8434 consensus_queue.cc:237] T 00000000000000000000000000000000 P 36274b6d931c4f47975a3dd9c329e22d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36274b6d931c4f47975a3dd9c329e22d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38001 } }
I20260811 05:17:47.187139 8423 sys_catalog.cc:564] T 00000000000000000000000000000000 P 36274b6d931c4f47975a3dd9c329e22d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:47.188400 8436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36274b6d931c4f47975a3dd9c329e22d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 36274b6d931c4f47975a3dd9c329e22d. Latest consensus state: current_term: 1 leader_uuid: "36274b6d931c4f47975a3dd9c329e22d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36274b6d931c4f47975a3dd9c329e22d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38001 } } }
I20260811 05:17:47.188553 8436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36274b6d931c4f47975a3dd9c329e22d [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:47.188944 8447 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:47.188992 8435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 36274b6d931c4f47975a3dd9c329e22d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "36274b6d931c4f47975a3dd9c329e22d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "36274b6d931c4f47975a3dd9c329e22d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38001 } } }
I20260811 05:17:47.189162 8435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 36274b6d931c4f47975a3dd9c329e22d [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:47.192233 8447 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:47.197819 8447 catalog_manager.cc:1347] Generated new cluster ID: 76c865d27f32435fac4ca1ab73c7df81
I20260811 05:17:47.197885 8447 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:47.206310 8447 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:47.206640 8447 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:47.214489 8447 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 36274b6d931c4f47975a3dd9c329e22d: Generated new TSK 0
I20260811 05:17:47.215196 8447 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:47.226613 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 8318
2026-08-11T05:17:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1663 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:47 dist-test-slave-04bb krb5kdc[8464](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:47 dist-test-slave-04bb krb5kdc[8464](info): set up 2 sockets
Aug 11 05:17:47 dist-test-slave-04bb krb5kdc[8464](info): commencing operation
krb5kdc: starting...
W20260811 05:17:48.513711 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.259s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:17:48 dist-test-slave-04bb krb5kdc[8464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:17:48.639425 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39801
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41627
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39801
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:48.752635 8483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:48.752880 8483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:48.752921 8483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:48.757095 8483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:48.757148 8483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:48.757167 8483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:48.757210 8483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:48.757246 8483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:48.762413 8483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41627
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:48.763447 8483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:48.764480 8483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:48.773155 8490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:48.773198 8483 server_base.cc:1034] running on GCE node
W20260811 05:17:48.773131 8494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:48.773124 8496 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:48.774012 8483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:48.775072 8483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:48.776289 8483 hybrid_clock.cc:648] HybridClock initialized: now 1786425468776260 us; error 40 us; skew 500 ppm
Aug 11 05:17:48 dist-test-slave-04bb krb5kdc[8464](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425468, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:48.779254 8483 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:17:48.780073 8483 webserver.cc:460] Webserver started at http://127.0.0.1:39781/ using document root <none> and password file <none>
I20260811 05:17:48.780669 8483 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:48.780736 8483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:48.780907 8483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:48.782914 8483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "46b3064fa7504a70bd92cae075c8f254"
format_stamp: "Formatted at 2026-08-11 05:17:48 on dist-test-slave-04bb"
server_key: "751a9684676e7c9cd8e6d2fd6c576ab2"
server_key_iv: "eb4aa6a5835fbde46e0bae3f1761236b"
server_key_version: "encryptionkey@0"
I20260811 05:17:48.783437 8483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "46b3064fa7504a70bd92cae075c8f254"
format_stamp: "Formatted at 2026-08-11 05:17:48 on dist-test-slave-04bb"
server_key: "751a9684676e7c9cd8e6d2fd6c576ab2"
server_key_iv: "eb4aa6a5835fbde46e0bae3f1761236b"
server_key_version: "encryptionkey@0"
I20260811 05:17:48.787272 8483 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 05:17:48.789791 8508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:48.791042 8483 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:17:48.791172 8483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "46b3064fa7504a70bd92cae075c8f254"
format_stamp: "Formatted at 2026-08-11 05:17:48 on dist-test-slave-04bb"
server_key: "751a9684676e7c9cd8e6d2fd6c576ab2"
server_key_iv: "eb4aa6a5835fbde46e0bae3f1761236b"
server_key_version: "encryptionkey@0"
I20260811 05:17:48.791307 8483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:48.816143 8483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:48.819630 8483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:48.819847 8483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:48.827591 8483 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39801
I20260811 05:17:48.827616 8589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39801 every 8 connection(s)
I20260811 05:17:48.828658 8483 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:48.831668 8591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:48.835994 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 8483
I20260811 05:17:48.836164 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:17:48.836544 7438 external_mini_cluster.cc:1428] Setting key 5f30bcae4d4456b6f2ccf8d7467d4098
I20260811 05:17:48.838409 8591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46b3064fa7504a70bd92cae075c8f254: Bootstrap starting.
I20260811 05:17:48.840858 8591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46b3064fa7504a70bd92cae075c8f254: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:48.841650 8591 log.cc:824] T 00000000000000000000000000000000 P 46b3064fa7504a70bd92cae075c8f254: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:48.843608 8591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46b3064fa7504a70bd92cae075c8f254: No bootstrap required, opened a new log
Aug 11 05:17:48 dist-test-slave-04bb krb5kdc[8464](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425468, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 05:17:48.847396 8591 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46b3064fa7504a70bd92cae075c8f254 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46b3064fa7504a70bd92cae075c8f254" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39801 } }
I20260811 05:17:48.847700 8591 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46b3064fa7504a70bd92cae075c8f254 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:48.847790 8591 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46b3064fa7504a70bd92cae075c8f254 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46b3064fa7504a70bd92cae075c8f254, State: Initialized, Role: FOLLOWER
I20260811 05:17:48.848337 8591 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46b3064fa7504a70bd92cae075c8f254 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46b3064fa7504a70bd92cae075c8f254" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39801 } }
I20260811 05:17:48.848508 8591 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46b3064fa7504a70bd92cae075c8f254 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:48.848618 8591 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46b3064fa7504a70bd92cae075c8f254 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:48.848778 8591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46b3064fa7504a70bd92cae075c8f254 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:48.849784 8591 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46b3064fa7504a70bd92cae075c8f254 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46b3064fa7504a70bd92cae075c8f254" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39801 } }
I20260811 05:17:48.850170 8591 leader_election.cc:302] T 00000000000000000000000000000000 P 46b3064fa7504a70bd92cae075c8f254 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46b3064fa7504a70bd92cae075c8f254; no voters:
I20260811 05:17:48.850562 8591 leader_election.cc:288] T 00000000000000000000000000000000 P 46b3064fa7504a70bd92cae075c8f254 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:48.850648 8600 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46b3064fa7504a70bd92cae075c8f254 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:48.850872 8600 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46b3064fa7504a70bd92cae075c8f254 [term 1 LEADER]: Becoming Leader. State: Replica: 46b3064fa7504a70bd92cae075c8f254, State: Running, Role: LEADER
I20260811 05:17:48.851284 8600 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46b3064fa7504a70bd92cae075c8f254 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46b3064fa7504a70bd92cae075c8f254" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39801 } }
I20260811 05:17:48.851625 8598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:48.838084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:41263 (local address 127.0.0.1:39801)
0811 05:17:48.838599 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:48.838616 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:48.838652 (+ 36us) server_negotiation.cc:407] Connection header received
0811 05:17:48.839274 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:48.839290 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:48.839647 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:48.839983 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:48.840863 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:48.841958 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:48.842821 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:48.843283 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:48.843463 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:48.846042 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:48.846048 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:48.846061 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:48.846088 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:48.848037 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:48.848584 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:48.848591 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:48.848596 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:48.848786 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:48.849162 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:48.849165 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:48.849167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:48.849594 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:48.849859 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:48.850339 (+ 480us) server_negotiation.cc:299] Negotiation successful
0811 05:17:48.850598 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":148,"threads_started":1}
I20260811 05:17:48.852066 8591 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46b3064fa7504a70bd92cae075c8f254 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:48.854596 8604 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46b3064fa7504a70bd92cae075c8f254 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46b3064fa7504a70bd92cae075c8f254. Latest consensus state: current_term: 1 leader_uuid: "46b3064fa7504a70bd92cae075c8f254" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46b3064fa7504a70bd92cae075c8f254" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39801 } } }
I20260811 05:17:48.854743 8604 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46b3064fa7504a70bd92cae075c8f254 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:48.855089 8605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46b3064fa7504a70bd92cae075c8f254 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46b3064fa7504a70bd92cae075c8f254" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46b3064fa7504a70bd92cae075c8f254" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39801 } } }
I20260811 05:17:48.855397 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:48.855520 8605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46b3064fa7504a70bd92cae075c8f254 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:48.855408 8624 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:48.859030 8624 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:48.865031 8624 catalog_manager.cc:1347] Generated new cluster ID: 02300dc0d4c14f84ba1d03450c527dfb
I20260811 05:17:48.865092 8624 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:48.871285 8598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:48.857821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:52269 (local address 127.0.0.1:39801)
0811 05:17:48.857960 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:48.857964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:48.858140 (+ 176us) server_negotiation.cc:407] Connection header received
0811 05:17:48.858369 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:48.858372 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:48.858423 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:48.858508 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:48.859459 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:48.860194 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:48.861502 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:48.861843 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:48.861950 (+ 107us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:48.863877 (+ 1927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:48.863880 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:48.863883 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:48.863912 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:48.869280 (+ 5368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:48.869892 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:48.869895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:48.869897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:48.869978 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:48.870374 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:48.870377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:48.870379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:48.870571 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:48.870692 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:48.871030 (+ 338us) server_negotiation.cc:299] Negotiation successful
0811 05:17:48.871132 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 05:17:48.873996 8624 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:48.874918 8624 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:48.880505 8624 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46b3064fa7504a70bd92cae075c8f254: Generated new TSK 0
I20260811 05:17:48.881188 8624 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:48.887584 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 8483
W20260811 05:17:48.908610 8462 connection.cc:535] client connection to 127.0.0.1:39801 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39801): Connection reset by peer (error 104)
2026-08-11T05:17:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1668 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:48 dist-test-slave-04bb krb5kdc[8641](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:48 dist-test-slave-04bb krb5kdc[8641](info): set up 2 sockets
Aug 11 05:17:48 dist-test-slave-04bb krb5kdc[8641](info): commencing operation
krb5kdc: starting...
W20260811 05:17:50.202450 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.279s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:17:50 dist-test-slave-04bb krb5kdc[8641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425470, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:17:50.330760 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33495
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:50.446952 8659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:50.447227 8659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:50.447273 8659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:50.451292 8659 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:50.451354 8659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:50.451377 8659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:50.451397 8659 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:50.451452 8659 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:50.456619 8659 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:50.457630 8659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:50.458647 8659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:50.467386 8667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:50.467521 8659 server_base.cc:1034] running on GCE node
W20260811 05:17:50.467406 8671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:50.467386 8677 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:50.468204 8659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:50.469344 8659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:50.470507 8659 hybrid_clock.cc:648] HybridClock initialized: now 1786425470470475 us; error 45 us; skew 500 ppm
Aug 11 05:17:50 dist-test-slave-04bb krb5kdc[8641](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425470, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:50.473206 8659 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:17:50.474028 8659 webserver.cc:460] Webserver started at http://127.0.0.1:40603/ using document root <none> and password file <none>
I20260811 05:17:50.474551 8659 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:50.474610 8659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:50.474782 8659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:50.476832 8659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "df5c9555e5904c3f9a2ca7adbff15b5f"
format_stamp: "Formatted at 2026-08-11 05:17:50 on dist-test-slave-04bb"
server_key: "109c6d14c3f1cda8509b506b0a3efaf2"
server_key_iv: "bb77f597ccb93f40232f44ba2269aba8"
server_key_version: "encryptionkey@0"
I20260811 05:17:50.477500 8659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "df5c9555e5904c3f9a2ca7adbff15b5f"
format_stamp: "Formatted at 2026-08-11 05:17:50 on dist-test-slave-04bb"
server_key: "109c6d14c3f1cda8509b506b0a3efaf2"
server_key_iv: "bb77f597ccb93f40232f44ba2269aba8"
server_key_version: "encryptionkey@0"
I20260811 05:17:50.481429 8659 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 05:17:50.483923 8687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:50.485194 8659 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:17:50.485345 8659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "df5c9555e5904c3f9a2ca7adbff15b5f"
format_stamp: "Formatted at 2026-08-11 05:17:50 on dist-test-slave-04bb"
server_key: "109c6d14c3f1cda8509b506b0a3efaf2"
server_key_iv: "bb77f597ccb93f40232f44ba2269aba8"
server_key_version: "encryptionkey@0"
I20260811 05:17:50.485440 8659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:50.512823 8659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:50.515827 8659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:50.516039 8659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:50.523303 8659 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33495
I20260811 05:17:50.523586 8770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33495 every 8 connection(s)
I20260811 05:17:50.524291 8659 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:50.527204 8772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:50.528142 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 8659
I20260811 05:17:50.528297 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:17:50.528653 7438 external_mini_cluster.cc:1428] Setting key 3ab6473ee9dbe7827ab17a412014d0d8
I20260811 05:17:50.533864 8772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df5c9555e5904c3f9a2ca7adbff15b5f: Bootstrap starting.
I20260811 05:17:50.536504 8772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df5c9555e5904c3f9a2ca7adbff15b5f: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:50.537429 8772 log.cc:824] T 00000000000000000000000000000000 P df5c9555e5904c3f9a2ca7adbff15b5f: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:50.538340 8776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:50.530087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:60731 (local address 127.0.0.1:33495)
0811 05:17:50.530500 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:50.530518 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:50.530567 (+ 49us) server_negotiation.cc:407] Connection header received
0811 05:17:50.531224 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:50.531241 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:50.531574 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:50.531857 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:50.532234 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:50.533284 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:50.534458 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:50.534999 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:50.535143 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:50.536142 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:50.536147 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:50.536162 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:50.536700 (+ 538us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:50.536780 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:50.536808 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:50.536924 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:50.537173 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 05:17:50.537331 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":142,"threads_started":1}
I20260811 05:17:50.539463 8772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df5c9555e5904c3f9a2ca7adbff15b5f: No bootstrap required, opened a new log
I20260811 05:17:50.541517 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:50.543737 8772 raft_consensus.cc:348] T 00000000000000000000000000000000 P df5c9555e5904c3f9a2ca7adbff15b5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df5c9555e5904c3f9a2ca7adbff15b5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33495 } }
I20260811 05:17:50.544075 8772 raft_consensus.cc:374] T 00000000000000000000000000000000 P df5c9555e5904c3f9a2ca7adbff15b5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:50.544152 8772 raft_consensus.cc:729] T 00000000000000000000000000000000 P df5c9555e5904c3f9a2ca7adbff15b5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df5c9555e5904c3f9a2ca7adbff15b5f, State: Initialized, Role: FOLLOWER
I20260811 05:17:50.544698 8772 consensus_queue.cc:260] T 00000000000000000000000000000000 P df5c9555e5904c3f9a2ca7adbff15b5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df5c9555e5904c3f9a2ca7adbff15b5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33495 } }
I20260811 05:17:50.544934 8772 raft_consensus.cc:388] T 00000000000000000000000000000000 P df5c9555e5904c3f9a2ca7adbff15b5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:50.545011 8772 raft_consensus.cc:482] T 00000000000000000000000000000000 P df5c9555e5904c3f9a2ca7adbff15b5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:50.545135 8772 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df5c9555e5904c3f9a2ca7adbff15b5f [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:50.546267 8772 raft_consensus.cc:504] T 00000000000000000000000000000000 P df5c9555e5904c3f9a2ca7adbff15b5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df5c9555e5904c3f9a2ca7adbff15b5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33495 } }
I20260811 05:17:50.546645 8772 leader_election.cc:302] T 00000000000000000000000000000000 P df5c9555e5904c3f9a2ca7adbff15b5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df5c9555e5904c3f9a2ca7adbff15b5f; no voters:
I20260811 05:17:50.547042 8772 leader_election.cc:288] T 00000000000000000000000000000000 P df5c9555e5904c3f9a2ca7adbff15b5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:50.547119 8790 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df5c9555e5904c3f9a2ca7adbff15b5f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:50.547403 8790 raft_consensus.cc:686] T 00000000000000000000000000000000 P df5c9555e5904c3f9a2ca7adbff15b5f [term 1 LEADER]: Becoming Leader. State: Replica: df5c9555e5904c3f9a2ca7adbff15b5f, State: Running, Role: LEADER
I20260811 05:17:50.547765 8790 consensus_queue.cc:237] T 00000000000000000000000000000000 P df5c9555e5904c3f9a2ca7adbff15b5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df5c9555e5904c3f9a2ca7adbff15b5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33495 } }
I20260811 05:17:50.548478 8772 sys_catalog.cc:564] T 00000000000000000000000000000000 P df5c9555e5904c3f9a2ca7adbff15b5f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:50.548872 8776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:50.543899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:54535 (local address 127.0.0.1:33495)
0811 05:17:50.544027 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:50.544034 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:50.544211 (+ 177us) server_negotiation.cc:407] Connection header received
0811 05:17:50.544342 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:50.544344 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:50.544408 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:50.544500 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:50.544823 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:50.545578 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:50.546691 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:50.547021 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:50.547142 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:50.548235 (+ 1093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:50.548267 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:50.548270 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:50.548291 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:50.548317 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:50.548328 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:50.548459 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:50.548687 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 05:17:50.548733 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 05:17:50.549048 8790 sys_catalog.cc:455] T 00000000000000000000000000000000 P df5c9555e5904c3f9a2ca7adbff15b5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader df5c9555e5904c3f9a2ca7adbff15b5f. Latest consensus state: current_term: 1 leader_uuid: "df5c9555e5904c3f9a2ca7adbff15b5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df5c9555e5904c3f9a2ca7adbff15b5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33495 } } }
I20260811 05:17:50.549170 8790 sys_catalog.cc:458] T 00000000000000000000000000000000 P df5c9555e5904c3f9a2ca7adbff15b5f [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:50.549857 8791 sys_catalog.cc:455] T 00000000000000000000000000000000 P df5c9555e5904c3f9a2ca7adbff15b5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df5c9555e5904c3f9a2ca7adbff15b5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df5c9555e5904c3f9a2ca7adbff15b5f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33495 } } }
I20260811 05:17:50.549975 8791 sys_catalog.cc:458] T 00000000000000000000000000000000 P df5c9555e5904c3f9a2ca7adbff15b5f [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:50.550324 8795 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:50.553586 8795 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:50.559276 8795 catalog_manager.cc:1347] Generated new cluster ID: c80709a860404d63a0db63e94806223b
I20260811 05:17:50.559347 8795 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:50.577832 8795 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:50.578889 8795 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:50.584084 8795 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df5c9555e5904c3f9a2ca7adbff15b5f: Generated new TSK 0
I20260811 05:17:50.585004 8795 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:50.595074 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 8659
W20260811 05:17:50.614895 8635 connection.cc:535] client connection to 127.0.0.1:33495 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33495): Connection reset by peer (error 104)
2026-08-11T05:17:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1706 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:50 dist-test-slave-04bb krb5kdc[8830](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:50 dist-test-slave-04bb krb5kdc[8830](info): set up 2 sockets
Aug 11 05:17:50 dist-test-slave-04bb krb5kdc[8830](info): commencing operation
krb5kdc: starting...
W20260811 05:17:51.874202 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:17:51 dist-test-slave-04bb krb5kdc[8830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:17:52.008116 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42223
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:52.126958 8845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:52.127262 8845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:52.127336 8845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:52.132086 8845 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:52.132144 8845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:52.132164 8845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:52.132180 8845 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:52.132196 8845 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:52.137466 8845 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:52.138572 8845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:52.139631 8845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:52.148545 8854 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:52.148567 8855 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:52.148545 8858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:52.148736 8845 server_base.cc:1034] running on GCE node
I20260811 05:17:52.149513 8845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:52.150617 8845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:52.151778 8845 hybrid_clock.cc:648] HybridClock initialized: now 1786425472151741 us; error 45 us; skew 500 ppm
Aug 11 05:17:52 dist-test-slave-04bb krb5kdc[8830](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425472, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:52.154573 8845 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:17:52.155476 8845 webserver.cc:460] Webserver started at http://127.0.0.1:45217/ using document root <none> and password file <none>
I20260811 05:17:52.156006 8845 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:52.156064 8845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:52.156234 8845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:52.158344 8845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "7f53ea44b4394998ba34211f48a73d2c"
format_stamp: "Formatted at 2026-08-11 05:17:52 on dist-test-slave-04bb"
server_key: "85e601a1259150137e3c55fab7baa9ee"
server_key_iv: "ad95a1b096dc5860ca23ee4d69a23218"
server_key_version: "encryptionkey@0"
I20260811 05:17:52.158874 8845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "7f53ea44b4394998ba34211f48a73d2c"
format_stamp: "Formatted at 2026-08-11 05:17:52 on dist-test-slave-04bb"
server_key: "85e601a1259150137e3c55fab7baa9ee"
server_key_iv: "ad95a1b096dc5860ca23ee4d69a23218"
server_key_version: "encryptionkey@0"
I20260811 05:17:52.162628 8845 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 05:17:52.165119 8871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:52.166337 8845 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:17:52.166440 8845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "7f53ea44b4394998ba34211f48a73d2c"
format_stamp: "Formatted at 2026-08-11 05:17:52 on dist-test-slave-04bb"
server_key: "85e601a1259150137e3c55fab7baa9ee"
server_key_iv: "ad95a1b096dc5860ca23ee4d69a23218"
server_key_version: "encryptionkey@0"
I20260811 05:17:52.166524 8845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:52.171149 8845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:52.174141 8845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:52.174291 8845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:52.181649 8845 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42223
I20260811 05:17:52.181667 8951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42223 every 8 connection(s)
I20260811 05:17:52.182595 8845 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:52.185012 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 8845
I20260811 05:17:52.185159 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:17:52.185492 7438 external_mini_cluster.cc:1428] Setting key afcc2b8b0fbb7a3954167fd09d9083c4
I20260811 05:17:52.185992 8952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:52.191011 8956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:52.187033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:48269 (local address 127.0.0.1:42223)
0811 05:17:52.187504 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:52.187518 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:52.187556 (+ 38us) server_negotiation.cc:407] Connection header received
0811 05:17:52.188226 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:52.188245 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:52.188605 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:52.188957 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:52.189822 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:52.189827 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:52.189839 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:52.189850 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:52.189904 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:52.189924 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:52.190077 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:52.190150 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 05:17:52.190318 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":148,"threads_started":1}
I20260811 05:17:52.192842 8952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f53ea44b4394998ba34211f48a73d2c: Bootstrap starting.
I20260811 05:17:52.193889 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:52.195382 8952 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f53ea44b4394998ba34211f48a73d2c: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:52.196264 8952 log.cc:824] T 00000000000000000000000000000000 P 7f53ea44b4394998ba34211f48a73d2c: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:52.197785 8956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:52.195955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:35895 (local address 127.0.0.1:42223)
0811 05:17:52.196070 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:52.196076 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:52.196255 (+ 179us) server_negotiation.cc:407] Connection header received
0811 05:17:52.196407 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:52.196410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:52.196453 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:52.196517 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:52.197368 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:52.197371 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:52.197373 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:52.197383 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:52.197405 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:52.197414 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:52.197483 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:52.197596 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 05:17:52.197628 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 05:17:52.198720 8952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f53ea44b4394998ba34211f48a73d2c: No bootstrap required, opened a new log
I20260811 05:17:52.202251 8952 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f53ea44b4394998ba34211f48a73d2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f53ea44b4394998ba34211f48a73d2c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42223 } }
I20260811 05:17:52.202469 8952 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f53ea44b4394998ba34211f48a73d2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:52.202550 8952 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f53ea44b4394998ba34211f48a73d2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f53ea44b4394998ba34211f48a73d2c, State: Initialized, Role: FOLLOWER
I20260811 05:17:52.202996 8952 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f53ea44b4394998ba34211f48a73d2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f53ea44b4394998ba34211f48a73d2c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42223 } }
I20260811 05:17:52.203143 8952 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f53ea44b4394998ba34211f48a73d2c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:52.203213 8952 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f53ea44b4394998ba34211f48a73d2c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:52.203348 8952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f53ea44b4394998ba34211f48a73d2c [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:52.204267 8952 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f53ea44b4394998ba34211f48a73d2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f53ea44b4394998ba34211f48a73d2c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42223 } }
I20260811 05:17:52.204672 8952 leader_election.cc:302] T 00000000000000000000000000000000 P 7f53ea44b4394998ba34211f48a73d2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f53ea44b4394998ba34211f48a73d2c; no voters:
I20260811 05:17:52.205044 8952 leader_election.cc:288] T 00000000000000000000000000000000 P 7f53ea44b4394998ba34211f48a73d2c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:52.205185 8968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f53ea44b4394998ba34211f48a73d2c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:52.205437 8968 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f53ea44b4394998ba34211f48a73d2c [term 1 LEADER]: Becoming Leader. State: Replica: 7f53ea44b4394998ba34211f48a73d2c, State: Running, Role: LEADER
I20260811 05:17:52.205809 8968 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f53ea44b4394998ba34211f48a73d2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f53ea44b4394998ba34211f48a73d2c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42223 } }
I20260811 05:17:52.206362 8952 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f53ea44b4394998ba34211f48a73d2c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:52.207690 8971 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f53ea44b4394998ba34211f48a73d2c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f53ea44b4394998ba34211f48a73d2c. Latest consensus state: current_term: 1 leader_uuid: "7f53ea44b4394998ba34211f48a73d2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f53ea44b4394998ba34211f48a73d2c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42223 } } }
I20260811 05:17:52.207743 8969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f53ea44b4394998ba34211f48a73d2c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f53ea44b4394998ba34211f48a73d2c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f53ea44b4394998ba34211f48a73d2c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42223 } } }
I20260811 05:17:52.207919 8971 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f53ea44b4394998ba34211f48a73d2c [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:52.207929 8969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f53ea44b4394998ba34211f48a73d2c [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:52.209022 8986 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:52.211596 8986 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:52.216629 8986 catalog_manager.cc:1347] Generated new cluster ID: 32e5562e0db940f78ff6a42af2123adc
I20260811 05:17:52.216692 8986 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:52.241911 8986 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:52.242303 8986 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:52.256913 8986 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f53ea44b4394998ba34211f48a73d2c: Generated new TSK 0
I20260811 05:17:52.257805 8986 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:52.268018 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 8845
2026-08-11T05:17:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1666 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:52 dist-test-slave-04bb krb5kdc[9008](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:52 dist-test-slave-04bb krb5kdc[9008](info): set up 2 sockets
Aug 11 05:17:52 dist-test-slave-04bb krb5kdc[9008](info): commencing operation
krb5kdc: starting...
W20260811 05:17:53.539017 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:17:53 dist-test-slave-04bb krb5kdc[9008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:17:53.671967 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35975
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35975
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:53.785847 9027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:53.786155 9027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:53.786240 9027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:53.790231 9027 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:53.790325 9027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:53.790370 9027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:53.790429 9027 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:53.790469 9027 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:53.795634 9027 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35975
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35975
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:53.796725 9027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:53.797766 9027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:53.806473 9037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:53.806548 9027 server_base.cc:1034] running on GCE node
W20260811 05:17:53.806511 9035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:53.806509 9034 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:53.807291 9027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:53.808408 9027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:53.809572 9027 hybrid_clock.cc:648] HybridClock initialized: now 1786425473809559 us; error 28 us; skew 500 ppm
Aug 11 05:17:53 dist-test-slave-04bb krb5kdc[9008](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425473, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:53.812312 9027 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:17:53.813271 9027 webserver.cc:460] Webserver started at http://127.0.0.1:33909/ using document root <none> and password file <none>
I20260811 05:17:53.813797 9027 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:53.813856 9027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:53.814025 9027 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:53.815997 9027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "a75592d7cf2d478b99bccc12901143e0"
format_stamp: "Formatted at 2026-08-11 05:17:53 on dist-test-slave-04bb"
server_key: "efe3025ebe0c70b890c8773ce14b2a17"
server_key_iv: "ae043e16df7ccb5deb224fc6ccd17be3"
server_key_version: "encryptionkey@0"
I20260811 05:17:53.816511 9027 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "a75592d7cf2d478b99bccc12901143e0"
format_stamp: "Formatted at 2026-08-11 05:17:53 on dist-test-slave-04bb"
server_key: "efe3025ebe0c70b890c8773ce14b2a17"
server_key_iv: "ae043e16df7ccb5deb224fc6ccd17be3"
server_key_version: "encryptionkey@0"
I20260811 05:17:53.820274 9027 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260811 05:17:53.822674 9046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:53.823896 9027 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:17:53.824049 9027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "a75592d7cf2d478b99bccc12901143e0"
format_stamp: "Formatted at 2026-08-11 05:17:53 on dist-test-slave-04bb"
server_key: "efe3025ebe0c70b890c8773ce14b2a17"
server_key_iv: "ae043e16df7ccb5deb224fc6ccd17be3"
server_key_version: "encryptionkey@0"
I20260811 05:17:53.824158 9027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:53.839692 9027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:53.843091 9027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:53.843328 9027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:53.850975 9027 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35975
I20260811 05:17:53.850988 9125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35975 every 8 connection(s)
I20260811 05:17:53.851944 9027 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:53.854836 9126 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:53.859279 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 9027
I20260811 05:17:53.859417 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:17:53.859702 7438 external_mini_cluster.cc:1428] Setting key c5c9287494265a92bae25d16cb61003d
I20260811 05:17:53.861147 9126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a75592d7cf2d478b99bccc12901143e0: Bootstrap starting.
I20260811 05:17:53.863615 9126 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a75592d7cf2d478b99bccc12901143e0: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:53.864457 9126 log.cc:824] T 00000000000000000000000000000000 P a75592d7cf2d478b99bccc12901143e0: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:53.866724 9126 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a75592d7cf2d478b99bccc12901143e0: No bootstrap required, opened a new log
Aug 11 05:17:53 dist-test-slave-04bb krb5kdc[9008](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425473, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 05:17:53.871657 9126 raft_consensus.cc:348] T 00000000000000000000000000000000 P a75592d7cf2d478b99bccc12901143e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a75592d7cf2d478b99bccc12901143e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35975 } }
I20260811 05:17:53.871963 9126 raft_consensus.cc:374] T 00000000000000000000000000000000 P a75592d7cf2d478b99bccc12901143e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:53.872032 9126 raft_consensus.cc:729] T 00000000000000000000000000000000 P a75592d7cf2d478b99bccc12901143e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a75592d7cf2d478b99bccc12901143e0, State: Initialized, Role: FOLLOWER
I20260811 05:17:53.872601 9126 consensus_queue.cc:260] T 00000000000000000000000000000000 P a75592d7cf2d478b99bccc12901143e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a75592d7cf2d478b99bccc12901143e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35975 } }
I20260811 05:17:53.872781 9126 raft_consensus.cc:388] T 00000000000000000000000000000000 P a75592d7cf2d478b99bccc12901143e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:53.872859 9126 raft_consensus.cc:482] T 00000000000000000000000000000000 P a75592d7cf2d478b99bccc12901143e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:53.872960 9126 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a75592d7cf2d478b99bccc12901143e0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:53.874013 9126 raft_consensus.cc:504] T 00000000000000000000000000000000 P a75592d7cf2d478b99bccc12901143e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a75592d7cf2d478b99bccc12901143e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35975 } }
I20260811 05:17:53.874423 9126 leader_election.cc:302] T 00000000000000000000000000000000 P a75592d7cf2d478b99bccc12901143e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a75592d7cf2d478b99bccc12901143e0; no voters:
I20260811 05:17:53.874805 9126 leader_election.cc:288] T 00000000000000000000000000000000 P a75592d7cf2d478b99bccc12901143e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:53.874913 9132 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a75592d7cf2d478b99bccc12901143e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:53.875154 9132 raft_consensus.cc:686] T 00000000000000000000000000000000 P a75592d7cf2d478b99bccc12901143e0 [term 1 LEADER]: Becoming Leader. State: Replica: a75592d7cf2d478b99bccc12901143e0, State: Running, Role: LEADER
I20260811 05:17:53.875542 9132 consensus_queue.cc:237] T 00000000000000000000000000000000 P a75592d7cf2d478b99bccc12901143e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a75592d7cf2d478b99bccc12901143e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35975 } }
I20260811 05:17:53.875921 9129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:53.861272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:51269 (local address 127.0.0.1:35975)
0811 05:17:53.861718 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:53.861737 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:53.861773 (+ 36us) server_negotiation.cc:407] Connection header received
0811 05:17:53.862398 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:53.862415 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:53.862759 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:53.863036 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:53.863991 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:53.865304 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:53.866454 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:53.866973 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:53.867092 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:53.869999 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:53.870008 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:53.870031 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:53.870071 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:53.872217 (+ 2146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:53.872838 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:53.872844 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:53.872852 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:53.873006 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:53.873399 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:53.873402 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:53.873403 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:53.873805 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:53.873978 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:53.874657 (+ 679us) server_negotiation.cc:299] Negotiation successful
0811 05:17:53.874911 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":138,"threads_started":1}
I20260811 05:17:53.876128 9126 sys_catalog.cc:564] T 00000000000000000000000000000000 P a75592d7cf2d478b99bccc12901143e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:53.877240 9137 sys_catalog.cc:455] T 00000000000000000000000000000000 P a75592d7cf2d478b99bccc12901143e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a75592d7cf2d478b99bccc12901143e0. Latest consensus state: current_term: 1 leader_uuid: "a75592d7cf2d478b99bccc12901143e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a75592d7cf2d478b99bccc12901143e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35975 } } }
I20260811 05:17:53.877360 9137 sys_catalog.cc:458] T 00000000000000000000000000000000 P a75592d7cf2d478b99bccc12901143e0 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:53.877692 9133 sys_catalog.cc:455] T 00000000000000000000000000000000 P a75592d7cf2d478b99bccc12901143e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a75592d7cf2d478b99bccc12901143e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a75592d7cf2d478b99bccc12901143e0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35975 } } }
I20260811 05:17:53.877759 9148 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:53.877770 9133 sys_catalog.cc:458] T 00000000000000000000000000000000 P a75592d7cf2d478b99bccc12901143e0 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:53.879485 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:53.881350 9148 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:53.887653 9148 catalog_manager.cc:1347] Generated new cluster ID: ff5bc687c98b4addacbd5c37e7f44e9d
I20260811 05:17:53.887727 9148 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:53.892390 9129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:53.882084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:40783 (local address 127.0.0.1:35975)
0811 05:17:53.882214 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:53.882219 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:53.882683 (+ 464us) server_negotiation.cc:407] Connection header received
0811 05:17:53.882888 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:53.882893 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:53.882944 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:53.882997 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:53.883945 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:53.884773 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:53.885965 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:53.886722 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:53.886848 (+ 126us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:53.888663 (+ 1815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:53.888667 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:53.888671 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:53.888698 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:53.890458 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:53.891005 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:53.891009 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:53.891011 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:53.891105 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:53.891455 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:53.891458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:53.891460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:53.891679 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:53.891844 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:53.892117 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 05:17:53.892221 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 05:17:53.899081 9148 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:53.899993 9148 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:53.908504 9148 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a75592d7cf2d478b99bccc12901143e0: Generated new TSK 0
I20260811 05:17:53.909264 9148 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:53.920897 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 9027
W20260811 05:17:53.938251 9006 connection.cc:535] client connection to 127.0.0.1:35975 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35975): Connection reset by peer (error 104)
2026-08-11T05:17:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1656 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:53 dist-test-slave-04bb krb5kdc[9175](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:53 dist-test-slave-04bb krb5kdc[9175](info): set up 2 sockets
Aug 11 05:17:53 dist-test-slave-04bb krb5kdc[9175](info): commencing operation
krb5kdc: starting...
W20260811 05:17:55.193507 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:17:55 dist-test-slave-04bb krb5kdc[9175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425475, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:17:55.327677 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36691
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40377
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36691
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:55.448360 9196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:55.448673 9196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:55.448747 9196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:55.452749 9196 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:55.452824 9196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:55.452857 9196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:55.452919 9196 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:55.452972 9196 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:55.458274 9196 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40377
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36691
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36691
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:55.459393 9196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:55.460475 9196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:55.469317 9206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:55.469326 9201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:55.469340 9202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:55.469718 9196 server_base.cc:1034] running on GCE node
I20260811 05:17:55.470330 9196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:55.471454 9196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:55.472621 9196 hybrid_clock.cc:648] HybridClock initialized: now 1786425475472604 us; error 31 us; skew 500 ppm
Aug 11 05:17:55 dist-test-slave-04bb krb5kdc[9175](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425475, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:55.475529 9196 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:17:55.476488 9196 webserver.cc:460] Webserver started at http://127.0.0.1:45089/ using document root <none> and password file <none>
I20260811 05:17:55.477097 9196 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:55.477190 9196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:55.477453 9196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:55.479548 9196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "b4e91453f0bb4e3c8b8aca35ff1a22dc"
format_stamp: "Formatted at 2026-08-11 05:17:55 on dist-test-slave-04bb"
server_key: "a6820fe236700115a8b0d3a1bdedc858"
server_key_iv: "ee3211187e189a839d0f769d938731b6"
server_key_version: "encryptionkey@0"
I20260811 05:17:55.480114 9196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "b4e91453f0bb4e3c8b8aca35ff1a22dc"
format_stamp: "Formatted at 2026-08-11 05:17:55 on dist-test-slave-04bb"
server_key: "a6820fe236700115a8b0d3a1bdedc858"
server_key_iv: "ee3211187e189a839d0f769d938731b6"
server_key_version: "encryptionkey@0"
I20260811 05:17:55.484145 9196 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 05:17:55.486794 9217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:55.488463 9196 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 05:17:55.488674 9196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "b4e91453f0bb4e3c8b8aca35ff1a22dc"
format_stamp: "Formatted at 2026-08-11 05:17:55 on dist-test-slave-04bb"
server_key: "a6820fe236700115a8b0d3a1bdedc858"
server_key_iv: "ee3211187e189a839d0f769d938731b6"
server_key_version: "encryptionkey@0"
I20260811 05:17:55.488807 9196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:55.502389 9196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:55.505707 9196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:55.505931 9196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:55.513742 9196 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36691
I20260811 05:17:55.513763 9300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36691 every 8 connection(s)
I20260811 05:17:55.514812 9196 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:55.517792 9302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:55.523452 9302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4e91453f0bb4e3c8b8aca35ff1a22dc: Bootstrap starting.
I20260811 05:17:55.524758 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 9196
I20260811 05:17:55.524912 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:17:55.525197 7438 external_mini_cluster.cc:1428] Setting key 8ca825c81c5a2b3f829af98b97c7e272
I20260811 05:17:55.526381 9302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4e91453f0bb4e3c8b8aca35ff1a22dc: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:55.527315 9302 log.cc:824] T 00000000000000000000000000000000 P b4e91453f0bb4e3c8b8aca35ff1a22dc: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:55.529615 9302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4e91453f0bb4e3c8b8aca35ff1a22dc: No bootstrap required, opened a new log
I20260811 05:17:55.533820 9302 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4e91453f0bb4e3c8b8aca35ff1a22dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4e91453f0bb4e3c8b8aca35ff1a22dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36691 } }
I20260811 05:17:55.534107 9302 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4e91453f0bb4e3c8b8aca35ff1a22dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:55.534162 9302 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4e91453f0bb4e3c8b8aca35ff1a22dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4e91453f0bb4e3c8b8aca35ff1a22dc, State: Initialized, Role: FOLLOWER
I20260811 05:17:55.534642 9302 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4e91453f0bb4e3c8b8aca35ff1a22dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4e91453f0bb4e3c8b8aca35ff1a22dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36691 } }
I20260811 05:17:55.534806 9302 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4e91453f0bb4e3c8b8aca35ff1a22dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:55.534873 9302 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4e91453f0bb4e3c8b8aca35ff1a22dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:55.534968 9302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4e91453f0bb4e3c8b8aca35ff1a22dc [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:55.535211 9308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:55.526386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:49715 (local address 127.0.0.1:36691)
0811 05:17:55.526868 (+ 482us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:55.526889 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:55.526932 (+ 43us) server_negotiation.cc:407] Connection header received
0811 05:17:55.527812 (+ 880us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:55.527833 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:55.528267 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:55.528654 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:55.529092 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:55.530563 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:55.531414 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:55.532138 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:55.532339 (+ 201us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:55.533172 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:55.533178 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:55.533196 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:55.533697 (+ 501us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:55.533766 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:55.533795 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:55.533943 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:55.534091 (+ 148us) server_negotiation.cc:299] Negotiation successful
0811 05:17:55.534237 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":134,"threads_started":1}
I20260811 05:17:55.535943 9302 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4e91453f0bb4e3c8b8aca35ff1a22dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4e91453f0bb4e3c8b8aca35ff1a22dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36691 } }
I20260811 05:17:55.536283 9302 leader_election.cc:302] T 00000000000000000000000000000000 P b4e91453f0bb4e3c8b8aca35ff1a22dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4e91453f0bb4e3c8b8aca35ff1a22dc; no voters:
I20260811 05:17:55.536640 9302 leader_election.cc:288] T 00000000000000000000000000000000 P b4e91453f0bb4e3c8b8aca35ff1a22dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:55.536712 9312 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4e91453f0bb4e3c8b8aca35ff1a22dc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:55.536939 9312 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4e91453f0bb4e3c8b8aca35ff1a22dc [term 1 LEADER]: Becoming Leader. State: Replica: b4e91453f0bb4e3c8b8aca35ff1a22dc, State: Running, Role: LEADER
I20260811 05:17:55.537252 9312 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4e91453f0bb4e3c8b8aca35ff1a22dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4e91453f0bb4e3c8b8aca35ff1a22dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36691 } }
I20260811 05:17:55.537947 9302 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4e91453f0bb4e3c8b8aca35ff1a22dc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:55.538499 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:55.538878 9314 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4e91453f0bb4e3c8b8aca35ff1a22dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4e91453f0bb4e3c8b8aca35ff1a22dc. Latest consensus state: current_term: 1 leader_uuid: "b4e91453f0bb4e3c8b8aca35ff1a22dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4e91453f0bb4e3c8b8aca35ff1a22dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36691 } } }
I20260811 05:17:55.539011 9314 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4e91453f0bb4e3c8b8aca35ff1a22dc [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:55.539373 9313 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4e91453f0bb4e3c8b8aca35ff1a22dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4e91453f0bb4e3c8b8aca35ff1a22dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4e91453f0bb4e3c8b8aca35ff1a22dc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36691 } } }
I20260811 05:17:55.539474 9313 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4e91453f0bb4e3c8b8aca35ff1a22dc [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:55.539741 9326 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:55.543533 9326 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:55.545811 9308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:55.540910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:42667 (local address 127.0.0.1:36691)
0811 05:17:55.541018 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:55.541022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:55.541204 (+ 182us) server_negotiation.cc:407] Connection header received
0811 05:17:55.541363 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:55.541368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:55.541414 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:55.541493 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:55.541814 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:55.542579 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:55.543724 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:55.544086 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:55.544165 (+ 79us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:55.545319 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:55.545323 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:55.545326 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:55.545339 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:55.545367 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:55.545380 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:55.545479 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:55.545652 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 05:17:55.545687 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 05:17:55.549535 9326 catalog_manager.cc:1347] Generated new cluster ID: 082bdf98563c49da9c700557025c7fe4
I20260811 05:17:55.549597 9326 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:55.555464 9326 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:55.556367 9326 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:55.562968 9326 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4e91453f0bb4e3c8b8aca35ff1a22dc: Generated new TSK 0
I20260811 05:17:55.563576 9326 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:55.567185 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 9196
W20260811 05:17:55.581324 9173 connection.cc:535] client connection to 127.0.0.1:36691 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36691): Connection reset by peer (error 104)
2026-08-11T05:17:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1643 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:55 dist-test-slave-04bb krb5kdc[9353](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:55 dist-test-slave-04bb krb5kdc[9353](info): set up 2 sockets
Aug 11 05:17:55 dist-test-slave-04bb krb5kdc[9353](info): commencing operation
krb5kdc: starting...
W20260811 05:17:56.825951 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:17:56 dist-test-slave-04bb krb5kdc[9353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425476, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 05:17:56.957312 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33849
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:57.074738 9377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:57.074998 9377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:57.075042 9377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:57.079010 9377 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:57.079066 9377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:57.079085 9377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:57.079101 9377 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:57.079116 9377 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:57.084365 9377 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33849
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:57.085427 9377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:57.086416 9377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:57.095830 9388 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:57.095868 9386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:57.096035 9377 server_base.cc:1034] running on GCE node
W20260811 05:17:57.095904 9390 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:57.096705 9377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:57.097810 9377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:57.099001 9377 hybrid_clock.cc:648] HybridClock initialized: now 1786425477098970 us; error 43 us; skew 500 ppm
Aug 11 05:17:57 dist-test-slave-04bb krb5kdc[9353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425477, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:57.101945 9377 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 05:17:57.102860 9377 webserver.cc:460] Webserver started at http://127.0.0.1:40473/ using document root <none> and password file <none>
I20260811 05:17:57.103430 9377 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:57.103525 9377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:57.103773 9377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:57.105916 9377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "adf21c4343584d929d0f4bed7a779bf7"
format_stamp: "Formatted at 2026-08-11 05:17:57 on dist-test-slave-04bb"
server_key: "c3dc4cd2d95fd1a9057d51dae554cf55"
server_key_iv: "79b3d3877c70bfdca8a6f070b58c7449"
server_key_version: "encryptionkey@0"
I20260811 05:17:57.106503 9377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "adf21c4343584d929d0f4bed7a779bf7"
format_stamp: "Formatted at 2026-08-11 05:17:57 on dist-test-slave-04bb"
server_key: "c3dc4cd2d95fd1a9057d51dae554cf55"
server_key_iv: "79b3d3877c70bfdca8a6f070b58c7449"
server_key_version: "encryptionkey@0"
I20260811 05:17:57.110394 9377 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 05:17:57.112843 9401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:57.114077 9377 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:17:57.114257 9377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "adf21c4343584d929d0f4bed7a779bf7"
format_stamp: "Formatted at 2026-08-11 05:17:57 on dist-test-slave-04bb"
server_key: "c3dc4cd2d95fd1a9057d51dae554cf55"
server_key_iv: "79b3d3877c70bfdca8a6f070b58c7449"
server_key_version: "encryptionkey@0"
I20260811 05:17:57.114374 9377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:57.119076 9377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:57.122321 9377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:57.122542 9377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:57.130193 9377 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33849
I20260811 05:17:57.130239 9479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33849 every 8 connection(s)
I20260811 05:17:57.131165 9377 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:57.134084 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 9377
I20260811 05:17:57.134219 9483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:57.134245 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:17:57.134567 7438 external_mini_cluster.cc:1428] Setting key e9f666f8f375fb832f577bf0cf7ee57f
I20260811 05:17:57.140192 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:57.135946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:52347 (local address 127.0.0.1:33849)
0811 05:17:57.136381 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:57.136396 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:57.136434 (+ 38us) server_negotiation.cc:407] Connection header received
0811 05:17:57.137469 (+ 1035us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:57.137491 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:57.137780 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:57.138063 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:57.138952 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:57.138957 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:57.138969 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:57.138980 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:57.139035 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:57.139054 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:57.139168 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:57.139305 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 05:17:57.139493 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":128,"threads_started":1}
I20260811 05:17:57.141356 9483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adf21c4343584d929d0f4bed7a779bf7: Bootstrap starting.
I20260811 05:17:57.143471 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:57.144076 9483 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P adf21c4343584d929d0f4bed7a779bf7: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:57.144970 9483 log.cc:824] T 00000000000000000000000000000000 P adf21c4343584d929d0f4bed7a779bf7: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:57.147199 9483 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P adf21c4343584d929d0f4bed7a779bf7: No bootstrap required, opened a new log
I20260811 05:17:57.147491 9487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:57.145751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.220:55391 (local address 127.0.0.1:33849)
0811 05:17:57.145872 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:57.145878 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:57.146101 (+ 223us) server_negotiation.cc:407] Connection header received
0811 05:17:57.146191 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:57.146194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:57.146229 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:57.146306 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:57.147001 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:57.147004 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:57.147007 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:17:57.147019 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:57.147045 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:17:57.147059 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:57.147151 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:57.147332 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 05:17:57.147382 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 05:17:57.151016 9483 raft_consensus.cc:348] T 00000000000000000000000000000000 P adf21c4343584d929d0f4bed7a779bf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adf21c4343584d929d0f4bed7a779bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33849 } }
I20260811 05:17:57.151234 9483 raft_consensus.cc:374] T 00000000000000000000000000000000 P adf21c4343584d929d0f4bed7a779bf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:57.151279 9483 raft_consensus.cc:729] T 00000000000000000000000000000000 P adf21c4343584d929d0f4bed7a779bf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adf21c4343584d929d0f4bed7a779bf7, State: Initialized, Role: FOLLOWER
I20260811 05:17:57.151644 9483 consensus_queue.cc:260] T 00000000000000000000000000000000 P adf21c4343584d929d0f4bed7a779bf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adf21c4343584d929d0f4bed7a779bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33849 } }
I20260811 05:17:57.151755 9483 raft_consensus.cc:388] T 00000000000000000000000000000000 P adf21c4343584d929d0f4bed7a779bf7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:57.151800 9483 raft_consensus.cc:482] T 00000000000000000000000000000000 P adf21c4343584d929d0f4bed7a779bf7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:57.151880 9483 raft_consensus.cc:3037] T 00000000000000000000000000000000 P adf21c4343584d929d0f4bed7a779bf7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:57.152786 9483 raft_consensus.cc:504] T 00000000000000000000000000000000 P adf21c4343584d929d0f4bed7a779bf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adf21c4343584d929d0f4bed7a779bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33849 } }
I20260811 05:17:57.153141 9483 leader_election.cc:302] T 00000000000000000000000000000000 P adf21c4343584d929d0f4bed7a779bf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: adf21c4343584d929d0f4bed7a779bf7; no voters:
I20260811 05:17:57.153558 9483 leader_election.cc:288] T 00000000000000000000000000000000 P adf21c4343584d929d0f4bed7a779bf7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:57.153627 9495 raft_consensus.cc:2781] T 00000000000000000000000000000000 P adf21c4343584d929d0f4bed7a779bf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:57.153844 9495 raft_consensus.cc:686] T 00000000000000000000000000000000 P adf21c4343584d929d0f4bed7a779bf7 [term 1 LEADER]: Becoming Leader. State: Replica: adf21c4343584d929d0f4bed7a779bf7, State: Running, Role: LEADER
I20260811 05:17:57.154146 9495 consensus_queue.cc:237] T 00000000000000000000000000000000 P adf21c4343584d929d0f4bed7a779bf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adf21c4343584d929d0f4bed7a779bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33849 } }
I20260811 05:17:57.154908 9483 sys_catalog.cc:564] T 00000000000000000000000000000000 P adf21c4343584d929d0f4bed7a779bf7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:57.155977 9496 sys_catalog.cc:455] T 00000000000000000000000000000000 P adf21c4343584d929d0f4bed7a779bf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "adf21c4343584d929d0f4bed7a779bf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adf21c4343584d929d0f4bed7a779bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33849 } } }
I20260811 05:17:57.156113 9496 sys_catalog.cc:458] T 00000000000000000000000000000000 P adf21c4343584d929d0f4bed7a779bf7 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:57.156414 9497 sys_catalog.cc:455] T 00000000000000000000000000000000 P adf21c4343584d929d0f4bed7a779bf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader adf21c4343584d929d0f4bed7a779bf7. Latest consensus state: current_term: 1 leader_uuid: "adf21c4343584d929d0f4bed7a779bf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "adf21c4343584d929d0f4bed7a779bf7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33849 } } }
I20260811 05:17:57.156584 9497 sys_catalog.cc:458] T 00000000000000000000000000000000 P adf21c4343584d929d0f4bed7a779bf7 [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:57.156585 9510 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:57.160008 9510 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:57.165652 9510 catalog_manager.cc:1347] Generated new cluster ID: 9e4bd0100c234d429858e0d95e9f1cfa
I20260811 05:17:57.165766 9510 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:57.176868 9510 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:57.177181 9510 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:57.185820 9510 catalog_manager.cc:5868] T 00000000000000000000000000000000 P adf21c4343584d929d0f4bed7a779bf7: Generated new TSK 0
I20260811 05:17:57.186486 9510 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:57.192216 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 9377
2026-08-11T05:17:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1628 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:57 dist-test-slave-04bb krb5kdc[9532](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:57 dist-test-slave-04bb krb5kdc[9532](info): set up 2 sockets
Aug 11 05:17:57 dist-test-slave-04bb krb5kdc[9532](info): commencing operation
krb5kdc: starting...
W20260811 05:17:58.473644 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.249s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:17:58 dist-test-slave-04bb krb5kdc[9532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:17:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:17:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:17:58.604065 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:37129
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:35125
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:37129
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:17:58.718748 9555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:17:58.719069 9555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:17:58.719156 9555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:17:58.723157 9555 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:17:58.723253 9555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:17:58.723291 9555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:17:58.723348 9555 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:17:58.723384 9555 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:17:58.728765 9555 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:35125
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:37129
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:37129
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:17:58.729851 9555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:17:58.730888 9555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:17:58.739500 9567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:58.739580 9565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:17:58.739549 9564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:17:58.739760 9555 server_base.cc:1034] running on GCE node
I20260811 05:17:58.740595 9555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:17:58.741809 9555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:17:58.742992 9555 hybrid_clock.cc:648] HybridClock initialized: now 1786425478742962 us; error 40 us; skew 500 ppm
Aug 11 05:17:58 dist-test-slave-04bb krb5kdc[9532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425478, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:17:58.746124 9555 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:17:58.747103 9555 webserver.cc:460] Webserver started at http://127.7.67.190:45505/ using document root <none> and password file <none>
I20260811 05:17:58.747668 9555 fs_manager.cc:359] Metadata directory not provided
I20260811 05:17:58.747745 9555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:17:58.747944 9555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:17:58.750273 9555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "7bd6aabc7ba344029e6339fe80eb02dd"
format_stamp: "Formatted at 2026-08-11 05:17:58 on dist-test-slave-04bb"
server_key: "fb0ac7e4fc5935467f6499aacb57a471"
server_key_iv: "d45fc79757b6d91a8e926cd6545f02e5"
server_key_version: "encryptionkey@0"
I20260811 05:17:58.750846 9555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "7bd6aabc7ba344029e6339fe80eb02dd"
format_stamp: "Formatted at 2026-08-11 05:17:58 on dist-test-slave-04bb"
server_key: "fb0ac7e4fc5935467f6499aacb57a471"
server_key_iv: "d45fc79757b6d91a8e926cd6545f02e5"
server_key_version: "encryptionkey@0"
I20260811 05:17:58.754865 9555 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 05:17:58.757414 9580 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:17:58.758710 9555 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:17:58.758882 9555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "7bd6aabc7ba344029e6339fe80eb02dd"
format_stamp: "Formatted at 2026-08-11 05:17:58 on dist-test-slave-04bb"
server_key: "fb0ac7e4fc5935467f6499aacb57a471"
server_key_iv: "d45fc79757b6d91a8e926cd6545f02e5"
server_key_version: "encryptionkey@0"
I20260811 05:17:58.758999 9555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:17:58.773264 9555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:17:58.776877 9555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:17:58.777107 9555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:17:58.784476 9555 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:37129
I20260811 05:17:58.784564 9660 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:37129 every 8 connection(s)
I20260811 05:17:58.785571 9555 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:17:58.788713 9661 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:17:58.791338 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 9555
I20260811 05:17:58.791499 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:17:58.791769 7438 external_mini_cluster.cc:1428] Setting key d120edced6731f6c554eb380e17d8e5b
I20260811 05:17:58.795686 9661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bd6aabc7ba344029e6339fe80eb02dd: Bootstrap starting.
I20260811 05:17:58.798574 9661 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7bd6aabc7ba344029e6339fe80eb02dd: Neither blocks nor log segments found. Creating new log.
I20260811 05:17:58.799540 9661 log.cc:824] T 00000000000000000000000000000000 P 7bd6aabc7ba344029e6339fe80eb02dd: Log is configured to *not* fsync() on all Append() calls
I20260811 05:17:58.801980 9661 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bd6aabc7ba344029e6339fe80eb02dd: No bootstrap required, opened a new log
Aug 11 05:17:58 dist-test-slave-04bb krb5kdc[9532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425478, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:17:58.805505 9661 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7bd6aabc7ba344029e6339fe80eb02dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bd6aabc7ba344029e6339fe80eb02dd" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37129 } }
I20260811 05:17:58.805738 9661 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7bd6aabc7ba344029e6339fe80eb02dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:17:58.805832 9661 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7bd6aabc7ba344029e6339fe80eb02dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bd6aabc7ba344029e6339fe80eb02dd, State: Initialized, Role: FOLLOWER
I20260811 05:17:58.806391 9661 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bd6aabc7ba344029e6339fe80eb02dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bd6aabc7ba344029e6339fe80eb02dd" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37129 } }
I20260811 05:17:58.806564 9661 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7bd6aabc7ba344029e6339fe80eb02dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:17:58.806661 9661 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bd6aabc7ba344029e6339fe80eb02dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:17:58.806782 9661 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7bd6aabc7ba344029e6339fe80eb02dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:17:58.807920 9661 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bd6aabc7ba344029e6339fe80eb02dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bd6aabc7ba344029e6339fe80eb02dd" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37129 } }
I20260811 05:17:58.808259 9665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:58.793347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36195 (local address 127.7.67.190:37129)
0811 05:17:58.793820 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:58.793848 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:58.793896 (+ 48us) server_negotiation.cc:407] Connection header received
0811 05:17:58.794827 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:58.794854 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:58.795315 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:58.795713 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:58.796495 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:58.798047 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:58.799003 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:58.799733 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:58.799905 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:58.802772 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:58.802778 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:58.802792 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:58.802820 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:58.805090 (+ 2270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:58.805630 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:58.805635 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:58.805639 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:58.805800 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:58.806170 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:58.806174 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:58.806176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:58.806617 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:58.806887 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:58.807191 (+ 304us) server_negotiation.cc:299] Negotiation successful
0811 05:17:58.807428 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":125,"threads_started":1}
I20260811 05:17:58.808296 9661 leader_election.cc:302] T 00000000000000000000000000000000 P 7bd6aabc7ba344029e6339fe80eb02dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7bd6aabc7ba344029e6339fe80eb02dd; no voters:
I20260811 05:17:58.808750 9661 leader_election.cc:288] T 00000000000000000000000000000000 P 7bd6aabc7ba344029e6339fe80eb02dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:17:58.808852 9669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7bd6aabc7ba344029e6339fe80eb02dd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:17:58.809041 9669 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7bd6aabc7ba344029e6339fe80eb02dd [term 1 LEADER]: Becoming Leader. State: Replica: 7bd6aabc7ba344029e6339fe80eb02dd, State: Running, Role: LEADER
I20260811 05:17:58.809429 9669 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7bd6aabc7ba344029e6339fe80eb02dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bd6aabc7ba344029e6339fe80eb02dd" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37129 } }
I20260811 05:17:58.810153 9661 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7bd6aabc7ba344029e6339fe80eb02dd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:17:58.811312 9670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bd6aabc7ba344029e6339fe80eb02dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7bd6aabc7ba344029e6339fe80eb02dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bd6aabc7ba344029e6339fe80eb02dd" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37129 } } }
I20260811 05:17:58.811453 9670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bd6aabc7ba344029e6339fe80eb02dd [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:58.811718 9670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bd6aabc7ba344029e6339fe80eb02dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7bd6aabc7ba344029e6339fe80eb02dd. Latest consensus state: current_term: 1 leader_uuid: "7bd6aabc7ba344029e6339fe80eb02dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bd6aabc7ba344029e6339fe80eb02dd" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37129 } } }
I20260811 05:17:58.811811 9670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bd6aabc7ba344029e6339fe80eb02dd [sys.catalog]: This master's current role is: LEADER
I20260811 05:17:58.812003 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:17:58.812991 9683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:17:58.815871 9683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:17:58.822045 9683 catalog_manager.cc:1347] Generated new cluster ID: 1de6a43e742d4faf9e583ff417738f7d
I20260811 05:17:58.822223 9683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:17:58.824486 9665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:17:58.814740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36007 (local address 127.7.67.190:37129)
0811 05:17:58.814857 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 05:17:58.814861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:17:58.814874 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:17:58.814909 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:17:58.814912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:17:58.814952 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:17:58.815012 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:17:58.815963 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:58.816771 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:58.817945 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:17:58.818314 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:17:58.818409 (+ 95us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:17:58.820358 (+ 1949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:17:58.820361 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:17:58.820363 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:17:58.820386 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:17:58.822317 (+ 1931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:58.822881 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:58.822885 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:58.822887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:58.822990 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:17:58.823499 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:17:58.823503 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:17:58.823505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:17:58.823737 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:17:58.823903 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 05:17:58.824217 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 05:17:58.824319 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 05:17:58.840672 9683 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:17:58.841869 9683 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:17:58.847831 9683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7bd6aabc7ba344029e6339fe80eb02dd: Generated new TSK 0
I20260811 05:17:58.848727 9683 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:17:58.862505 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 9555
W20260811 05:17:58.870440 9530 connection.cc:535] client connection to 127.7.67.190:37129 recv error: Network error: failed to read from TLS socket (remote: 127.7.67.190:37129): Connection reset by peer (error 104)
2026-08-11T05:17:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1660 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:17:58 dist-test-slave-04bb krb5kdc[9714](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:17:58 dist-test-slave-04bb krb5kdc[9714](info): set up 2 sockets
Aug 11 05:17:58 dist-test-slave-04bb krb5kdc[9714](info): commencing operation
krb5kdc: starting...
W20260811 05:18:00.124415 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.239s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:18:00 dist-test-slave-04bb krb5kdc[9714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:18:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:18:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:18:00.255277 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:34741
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:33507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:34741
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:18:00.369526 9732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:18:00.369831 9732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:18:00.369936 9732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:18:00.375530 9732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:18:00.375598 9732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:18:00.375618 9732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:18:00.375635 9732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:18:00.375650 9732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:18:00.381043 9732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:33507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:34741
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:34741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:18:00.382118 9732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:18:00.383167 9732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:18:00.392256 9751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:18:00.392303 9749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:18:00.392251 9747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:18:00.392598 9732 server_base.cc:1034] running on GCE node
I20260811 05:18:00.393337 9732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:18:00.394443 9732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:18:00.395641 9732 hybrid_clock.cc:648] HybridClock initialized: now 1786425480395610 us; error 42 us; skew 500 ppm
Aug 11 05:18:00 dist-test-slave-04bb krb5kdc[9714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425480, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:18:00.398572 9732 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:18:00.399482 9732 webserver.cc:460] Webserver started at http://127.7.67.190:32905/ using document root <none> and password file <none>
I20260811 05:18:00.400060 9732 fs_manager.cc:359] Metadata directory not provided
I20260811 05:18:00.400158 9732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:18:00.400410 9732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:18:00.402582 9732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "0cc8149e74c24705b002940f3fc3cbdb"
format_stamp: "Formatted at 2026-08-11 05:18:00 on dist-test-slave-04bb"
server_key: "6bf69f06ce5a14733f2a76c749e90f61"
server_key_iv: "a506c07b697221073e82d1c24a8aa388"
server_key_version: "encryptionkey@0"
I20260811 05:18:00.403154 9732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "0cc8149e74c24705b002940f3fc3cbdb"
format_stamp: "Formatted at 2026-08-11 05:18:00 on dist-test-slave-04bb"
server_key: "6bf69f06ce5a14733f2a76c749e90f61"
server_key_iv: "a506c07b697221073e82d1c24a8aa388"
server_key_version: "encryptionkey@0"
I20260811 05:18:00.407019 9732 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 05:18:00.409641 9764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:18:00.410957 9732 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:18:00.411120 9732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "0cc8149e74c24705b002940f3fc3cbdb"
format_stamp: "Formatted at 2026-08-11 05:18:00 on dist-test-slave-04bb"
server_key: "6bf69f06ce5a14733f2a76c749e90f61"
server_key_iv: "a506c07b697221073e82d1c24a8aa388"
server_key_version: "encryptionkey@0"
I20260811 05:18:00.411242 9732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:18:00.426167 9732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:18:00.429595 9732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:18:00.429868 9732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:18:00.438513 9732 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:34741
I20260811 05:18:00.438526 9860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:34741 every 8 connection(s)
I20260811 05:18:00.439594 9732 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:18:00.442343 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 9732
I20260811 05:18:00.442516 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:18:00.442620 9861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:18:00.442888 7438 external_mini_cluster.cc:1428] Setting key 41dcb52ce4703e5915005ced63c3254b
I20260811 05:18:00.449702 9861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cc8149e74c24705b002940f3fc3cbdb: Bootstrap starting.
I20260811 05:18:00.452345 9861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0cc8149e74c24705b002940f3fc3cbdb: Neither blocks nor log segments found. Creating new log.
I20260811 05:18:00.452946 9866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:00.444402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39245 (local address 127.7.67.190:34741)
0811 05:18:00.444966 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:00.444984 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:00.445025 (+ 41us) server_negotiation.cc:407] Connection header received
0811 05:18:00.445978 (+ 953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:00.445998 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:00.446358 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:00.446673 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:00.447008 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:00.448138 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:00.449375 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:00.449961 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:00.450107 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:00.451084 (+ 977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:00.451090 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:00.451103 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:18:00.451492 (+ 389us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:00.451549 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:18:00.451572 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:00.451700 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:00.451984 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 05:18:00.452128 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":395,"thread_start_us":252,"threads_started":1}
I20260811 05:18:00.453229 9861 log.cc:824] T 00000000000000000000000000000000 P 0cc8149e74c24705b002940f3fc3cbdb: Log is configured to *not* fsync() on all Append() calls
I20260811 05:18:00.455546 9861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0cc8149e74c24705b002940f3fc3cbdb: No bootstrap required, opened a new log
I20260811 05:18:00.456163 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:18:00.459462 9861 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0cc8149e74c24705b002940f3fc3cbdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc8149e74c24705b002940f3fc3cbdb" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 34741 } }
I20260811 05:18:00.459717 9861 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0cc8149e74c24705b002940f3fc3cbdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:18:00.459808 9861 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0cc8149e74c24705b002940f3fc3cbdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cc8149e74c24705b002940f3fc3cbdb, State: Initialized, Role: FOLLOWER
I20260811 05:18:00.460292 9861 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0cc8149e74c24705b002940f3fc3cbdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc8149e74c24705b002940f3fc3cbdb" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 34741 } }
I20260811 05:18:00.460457 9861 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0cc8149e74c24705b002940f3fc3cbdb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:18:00.460565 9861 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0cc8149e74c24705b002940f3fc3cbdb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:18:00.460672 9861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0cc8149e74c24705b002940f3fc3cbdb [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:18:00.461593 9861 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0cc8149e74c24705b002940f3fc3cbdb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc8149e74c24705b002940f3fc3cbdb" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 34741 } }
I20260811 05:18:00.461951 9861 leader_election.cc:302] T 00000000000000000000000000000000 P 0cc8149e74c24705b002940f3fc3cbdb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0cc8149e74c24705b002940f3fc3cbdb; no voters:
I20260811 05:18:00.462285 9861 leader_election.cc:288] T 00000000000000000000000000000000 P 0cc8149e74c24705b002940f3fc3cbdb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:18:00.462381 9874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0cc8149e74c24705b002940f3fc3cbdb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:18:00.462564 9874 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0cc8149e74c24705b002940f3fc3cbdb [term 1 LEADER]: Becoming Leader. State: Replica: 0cc8149e74c24705b002940f3fc3cbdb, State: Running, Role: LEADER
I20260811 05:18:00.462879 9874 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0cc8149e74c24705b002940f3fc3cbdb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc8149e74c24705b002940f3fc3cbdb" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 34741 } }
I20260811 05:18:00.463117 9866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:00.458378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35807 (local address 127.7.67.190:34741)
0811 05:18:00.458508 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:00.458513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:00.458703 (+ 190us) server_negotiation.cc:407] Connection header received
0811 05:18:00.458877 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:00.458885 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:00.458926 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:00.459017 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:00.459324 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:00.460017 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:00.461132 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:00.461408 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:00.461529 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:00.462568 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:00.462572 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:00.462575 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:18:00.462589 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:00.462646 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:18:00.462673 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:00.462774 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:00.462910 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 05:18:00.462947 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 05:18:00.463636 9861 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0cc8149e74c24705b002940f3fc3cbdb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:18:00.464246 9874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cc8149e74c24705b002940f3fc3cbdb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0cc8149e74c24705b002940f3fc3cbdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc8149e74c24705b002940f3fc3cbdb" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 34741 } } }
I20260811 05:18:00.464413 9874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cc8149e74c24705b002940f3fc3cbdb [sys.catalog]: This master's current role is: LEADER
I20260811 05:18:00.464694 9874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0cc8149e74c24705b002940f3fc3cbdb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0cc8149e74c24705b002940f3fc3cbdb. Latest consensus state: current_term: 1 leader_uuid: "0cc8149e74c24705b002940f3fc3cbdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0cc8149e74c24705b002940f3fc3cbdb" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 34741 } } }
I20260811 05:18:00.464872 9874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0cc8149e74c24705b002940f3fc3cbdb [sys.catalog]: This master's current role is: LEADER
I20260811 05:18:00.465173 9883 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:18:00.468568 9883 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:18:00.474649 9883 catalog_manager.cc:1347] Generated new cluster ID: 713cc855f9e74bd2a8db135646e469e4
I20260811 05:18:00.474718 9883 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:18:00.490485 9883 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:18:00.491683 9883 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:18:00.497189 9883 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0cc8149e74c24705b002940f3fc3cbdb: Generated new TSK 0
I20260811 05:18:00.497917 9883 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:18:00.510164 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 9732
W20260811 05:18:00.525727 9712 connection.cc:535] client connection to 127.7.67.190:34741 recv error: Network error: failed to read from TLS socket (remote: 127.7.67.190:34741): Connection reset by peer (error 104)
2026-08-11T05:18:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1655 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:18:00 dist-test-slave-04bb krb5kdc[9906](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:18:00 dist-test-slave-04bb krb5kdc[9906](info): set up 2 sockets
Aug 11 05:18:00 dist-test-slave-04bb krb5kdc[9906](info): commencing operation
krb5kdc: starting...
W20260811 05:18:01.813373 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:18:01 dist-test-slave-04bb krb5kdc[9906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425481, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:18:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:18:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:18:01.947781 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:36719
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:35171
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:36719
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:18:02.063176 9941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:18:02.063493 9941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:18:02.063582 9941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:18:02.067682 9941 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:18:02.067785 9941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:18:02.067822 9941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:18:02.067883 9941 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:18:02.067939 9941 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:18:02.073318 9941 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:35171
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:36719
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:36719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:18:02.074477 9941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:18:02.075582 9941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:18:02.084419 9951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:18:02.084623 9941 server_base.cc:1034] running on GCE node
W20260811 05:18:02.084419 9954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:18:02.084481 9948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:18:02.085403 9941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:18:02.086534 9941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:18:02.087728 9941 hybrid_clock.cc:648] HybridClock initialized: now 1786425482087714 us; error 39 us; skew 500 ppm
Aug 11 05:18:02 dist-test-slave-04bb krb5kdc[9906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425482, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:18:02.090690 9941 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:18:02.091717 9941 webserver.cc:460] Webserver started at http://127.7.67.190:46513/ using document root <none> and password file <none>
I20260811 05:18:02.092307 9941 fs_manager.cc:359] Metadata directory not provided
I20260811 05:18:02.092412 9941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:18:02.092871 9941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:18:02.095556 9941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "b58e7bb098fe4cc4a25253936c707ba2"
format_stamp: "Formatted at 2026-08-11 05:18:02 on dist-test-slave-04bb"
server_key: "1335dba24df1e4c448651c1ad390fa0c"
server_key_iv: "c992583ced5239af48f1f7aa5a912444"
server_key_version: "encryptionkey@0"
I20260811 05:18:02.096330 9941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "b58e7bb098fe4cc4a25253936c707ba2"
format_stamp: "Formatted at 2026-08-11 05:18:02 on dist-test-slave-04bb"
server_key: "1335dba24df1e4c448651c1ad390fa0c"
server_key_iv: "c992583ced5239af48f1f7aa5a912444"
server_key_version: "encryptionkey@0"
I20260811 05:18:02.101289 9941 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 05:18:02.103649 9963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:18:02.104966 9941 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:18:02.105070 9941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "b58e7bb098fe4cc4a25253936c707ba2"
format_stamp: "Formatted at 2026-08-11 05:18:02 on dist-test-slave-04bb"
server_key: "1335dba24df1e4c448651c1ad390fa0c"
server_key_iv: "c992583ced5239af48f1f7aa5a912444"
server_key_version: "encryptionkey@0"
I20260811 05:18:02.105152 9941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:18:02.109536 9941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:18:02.112325 9941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:18:02.112473 9941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:18:02.119333 9941 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:36719
I20260811 05:18:02.119397 10048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:36719 every 8 connection(s)
I20260811 05:18:02.120329 9941 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:18:02.123323 10049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:18:02.124828 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 9941
I20260811 05:18:02.124955 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:18:02.125224 7438 external_mini_cluster.cc:1428] Setting key 391ff18867dbceee624f3630f9bad026
I20260811 05:18:02.129673 10049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b58e7bb098fe4cc4a25253936c707ba2: Bootstrap starting.
I20260811 05:18:02.130589 10054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:02.126734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33127 (local address 127.7.67.190:36719)
0811 05:18:02.127187 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:02.127203 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:02.127240 (+ 37us) server_negotiation.cc:407] Connection header received
0811 05:18:02.127905 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:02.127922 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:02.128194 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:02.128569 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:02.129384 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:02.129389 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:02.129401 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:18:02.129413 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:02.129470 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:18:02.129489 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:02.129644 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:02.129718 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 05:18:02.129874 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":135,"threads_started":1}
I20260811 05:18:02.132229 10049 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b58e7bb098fe4cc4a25253936c707ba2: Neither blocks nor log segments found. Creating new log.
I20260811 05:18:02.133119 10049 log.cc:824] T 00000000000000000000000000000000 P b58e7bb098fe4cc4a25253936c707ba2: Log is configured to *not* fsync() on all Append() calls
I20260811 05:18:02.133572 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:18:02.135202 10049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b58e7bb098fe4cc4a25253936c707ba2: No bootstrap required, opened a new log
I20260811 05:18:02.137387 10054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:02.135571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42807 (local address 127.7.67.190:36719)
0811 05:18:02.135689 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:02.135694 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:02.135901 (+ 207us) server_negotiation.cc:407] Connection header received
0811 05:18:02.136074 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:02.136077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:02.136113 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:02.136194 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:02.136962 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:02.136966 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:02.136969 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:18:02.136980 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:02.137007 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:18:02.137025 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:02.137116 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:02.137189 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 05:18:02.137228 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 05:18:02.139442 10049 raft_consensus.cc:348] T 00000000000000000000000000000000 P b58e7bb098fe4cc4a25253936c707ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58e7bb098fe4cc4a25253936c707ba2" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 36719 } }
I20260811 05:18:02.139657 10049 raft_consensus.cc:374] T 00000000000000000000000000000000 P b58e7bb098fe4cc4a25253936c707ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:18:02.139706 10049 raft_consensus.cc:729] T 00000000000000000000000000000000 P b58e7bb098fe4cc4a25253936c707ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b58e7bb098fe4cc4a25253936c707ba2, State: Initialized, Role: FOLLOWER
I20260811 05:18:02.140074 10049 consensus_queue.cc:260] T 00000000000000000000000000000000 P b58e7bb098fe4cc4a25253936c707ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58e7bb098fe4cc4a25253936c707ba2" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 36719 } }
I20260811 05:18:02.140188 10049 raft_consensus.cc:388] T 00000000000000000000000000000000 P b58e7bb098fe4cc4a25253936c707ba2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:18:02.140232 10049 raft_consensus.cc:482] T 00000000000000000000000000000000 P b58e7bb098fe4cc4a25253936c707ba2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:18:02.140301 10049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b58e7bb098fe4cc4a25253936c707ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:18:02.141196 10049 raft_consensus.cc:504] T 00000000000000000000000000000000 P b58e7bb098fe4cc4a25253936c707ba2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58e7bb098fe4cc4a25253936c707ba2" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 36719 } }
I20260811 05:18:02.141508 10049 leader_election.cc:302] T 00000000000000000000000000000000 P b58e7bb098fe4cc4a25253936c707ba2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b58e7bb098fe4cc4a25253936c707ba2; no voters:
I20260811 05:18:02.141767 10049 leader_election.cc:288] T 00000000000000000000000000000000 P b58e7bb098fe4cc4a25253936c707ba2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:18:02.141888 10068 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b58e7bb098fe4cc4a25253936c707ba2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:18:02.142158 10068 raft_consensus.cc:686] T 00000000000000000000000000000000 P b58e7bb098fe4cc4a25253936c707ba2 [term 1 LEADER]: Becoming Leader. State: Replica: b58e7bb098fe4cc4a25253936c707ba2, State: Running, Role: LEADER
I20260811 05:18:02.142485 10068 consensus_queue.cc:237] T 00000000000000000000000000000000 P b58e7bb098fe4cc4a25253936c707ba2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58e7bb098fe4cc4a25253936c707ba2" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 36719 } }
I20260811 05:18:02.143083 10049 sys_catalog.cc:564] T 00000000000000000000000000000000 P b58e7bb098fe4cc4a25253936c707ba2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:18:02.144419 10069 sys_catalog.cc:455] T 00000000000000000000000000000000 P b58e7bb098fe4cc4a25253936c707ba2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b58e7bb098fe4cc4a25253936c707ba2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58e7bb098fe4cc4a25253936c707ba2" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 36719 } } }
I20260811 05:18:02.144426 10070 sys_catalog.cc:455] T 00000000000000000000000000000000 P b58e7bb098fe4cc4a25253936c707ba2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b58e7bb098fe4cc4a25253936c707ba2. Latest consensus state: current_term: 1 leader_uuid: "b58e7bb098fe4cc4a25253936c707ba2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b58e7bb098fe4cc4a25253936c707ba2" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 36719 } } }
I20260811 05:18:02.144616 10069 sys_catalog.cc:458] T 00000000000000000000000000000000 P b58e7bb098fe4cc4a25253936c707ba2 [sys.catalog]: This master's current role is: LEADER
I20260811 05:18:02.144621 10070 sys_catalog.cc:458] T 00000000000000000000000000000000 P b58e7bb098fe4cc4a25253936c707ba2 [sys.catalog]: This master's current role is: LEADER
I20260811 05:18:02.145049 10082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:18:02.147857 10082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:18:02.153576 10082 catalog_manager.cc:1347] Generated new cluster ID: a1789eb52af7467db68949810ace499c
I20260811 05:18:02.153656 10082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:18:02.162312 10082 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:18:02.162616 10082 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:18:02.169731 10082 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b58e7bb098fe4cc4a25253936c707ba2: Generated new TSK 0
I20260811 05:18:02.170589 10082 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:18:02.178308 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 9941
2026-08-11T05:18:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1669 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:18:02 dist-test-slave-04bb krb5kdc[10099](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:18:02 dist-test-slave-04bb krb5kdc[10099](info): set up 2 sockets
Aug 11 05:18:02 dist-test-slave-04bb krb5kdc[10099](info): commencing operation
krb5kdc: starting...
W20260811 05:18:03.472162 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:18:03 dist-test-slave-04bb krb5kdc[10099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425483, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:18:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:18:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:18:03.599383 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:44517
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:44517
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:18:03.714381 10121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:18:03.714639 10121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:18:03.714680 10121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:18:03.718734 10121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:18:03.718797 10121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:18:03.718816 10121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:18:03.718833 10121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:18:03.718892 10121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:18:03.724133 10121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:44517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:44517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:18:03.725306 10121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:18:03.726377 10121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:18:03.735450 10131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:18:03.735600 10121 server_base.cc:1034] running on GCE node
W20260811 05:18:03.735466 10127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:18:03.735466 10129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:18:03.736380 10121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:18:03.737576 10121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:18:03.738785 10121 hybrid_clock.cc:648] HybridClock initialized: now 1786425483738765 us; error 33 us; skew 500 ppm
Aug 11 05:18:03 dist-test-slave-04bb krb5kdc[10099](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425483, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:18:03.741650 10121 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:18:03.742628 10121 webserver.cc:460] Webserver started at http://127.7.67.190:33279/ using document root <none> and password file <none>
I20260811 05:18:03.743218 10121 fs_manager.cc:359] Metadata directory not provided
I20260811 05:18:03.743314 10121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:18:03.743557 10121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:18:03.745683 10121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "0610d500f32c4b83abc4e635c11298a0"
format_stamp: "Formatted at 2026-08-11 05:18:03 on dist-test-slave-04bb"
server_key: "2ad53586adf72ef49f0ad2c535cf78eb"
server_key_iv: "416c15a199480f71f70ec8bbe40434fc"
server_key_version: "encryptionkey@0"
I20260811 05:18:03.746260 10121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "0610d500f32c4b83abc4e635c11298a0"
format_stamp: "Formatted at 2026-08-11 05:18:03 on dist-test-slave-04bb"
server_key: "2ad53586adf72ef49f0ad2c535cf78eb"
server_key_iv: "416c15a199480f71f70ec8bbe40434fc"
server_key_version: "encryptionkey@0"
I20260811 05:18:03.749979 10121 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 05:18:03.752410 10141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:18:03.753746 10121 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:18:03.753902 10121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "0610d500f32c4b83abc4e635c11298a0"
format_stamp: "Formatted at 2026-08-11 05:18:03 on dist-test-slave-04bb"
server_key: "2ad53586adf72ef49f0ad2c535cf78eb"
server_key_iv: "416c15a199480f71f70ec8bbe40434fc"
server_key_version: "encryptionkey@0"
I20260811 05:18:03.754015 10121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:18:03.779685 10121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:18:03.782723 10121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:18:03.782934 10121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:18:03.790414 10121 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:44517
I20260811 05:18:03.790441 10219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:44517 every 8 connection(s)
I20260811 05:18:03.791388 10121 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:18:03.794339 10220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:18:03.795771 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 10121
I20260811 05:18:03.795938 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:18:03.796272 7438 external_mini_cluster.cc:1428] Setting key 00ff1fac87dd04deb520f8ef1fe552c1
I20260811 05:18:03.801517 10220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0610d500f32c4b83abc4e635c11298a0: Bootstrap starting.
I20260811 05:18:03.804265 10220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0610d500f32c4b83abc4e635c11298a0: Neither blocks nor log segments found. Creating new log.
I20260811 05:18:03.805111 10220 log.cc:824] T 00000000000000000000000000000000 P 0610d500f32c4b83abc4e635c11298a0: Log is configured to *not* fsync() on all Append() calls
Aug 11 05:18:03 dist-test-slave-04bb krb5kdc[10099](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425483, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:18:03.807493 10220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0610d500f32c4b83abc4e635c11298a0: No bootstrap required, opened a new log
I20260811 05:18:03.812628 10223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:03.797815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38793 (local address 127.7.67.190:44517)
0811 05:18:03.798265 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:03.798286 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:03.798332 (+ 46us) server_negotiation.cc:407] Connection header received
0811 05:18:03.799168 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:03.799194 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:03.799647 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:03.800058 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:03.800826 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:03.802277 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:03.803129 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:03.803824 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:03.803999 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:03.807004 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:03.807010 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:03.807023 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:03.807049 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:03.809203 (+ 2154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:03.809793 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:03.809803 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:03.809812 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:03.809995 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:03.810286 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:03.810291 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:03.810294 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:03.810871 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:03.811066 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:03.811276 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 05:18:03.811585 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":115,"threads_started":1}
I20260811 05:18:03.812505 10220 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0610d500f32c4b83abc4e635c11298a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0610d500f32c4b83abc4e635c11298a0" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 44517 } }
I20260811 05:18:03.812790 10220 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0610d500f32c4b83abc4e635c11298a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:18:03.812865 10220 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0610d500f32c4b83abc4e635c11298a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0610d500f32c4b83abc4e635c11298a0, State: Initialized, Role: FOLLOWER
I20260811 05:18:03.813283 10220 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0610d500f32c4b83abc4e635c11298a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0610d500f32c4b83abc4e635c11298a0" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 44517 } }
I20260811 05:18:03.813417 10220 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0610d500f32c4b83abc4e635c11298a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:18:03.813503 10220 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0610d500f32c4b83abc4e635c11298a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:18:03.813606 10220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0610d500f32c4b83abc4e635c11298a0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:18:03.814775 10220 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0610d500f32c4b83abc4e635c11298a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0610d500f32c4b83abc4e635c11298a0" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 44517 } }
I20260811 05:18:03.815155 10220 leader_election.cc:302] T 00000000000000000000000000000000 P 0610d500f32c4b83abc4e635c11298a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0610d500f32c4b83abc4e635c11298a0; no voters:
I20260811 05:18:03.815578 10220 leader_election.cc:288] T 00000000000000000000000000000000 P 0610d500f32c4b83abc4e635c11298a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:18:03.815621 10228 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0610d500f32c4b83abc4e635c11298a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:18:03.815889 10228 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0610d500f32c4b83abc4e635c11298a0 [term 1 LEADER]: Becoming Leader. State: Replica: 0610d500f32c4b83abc4e635c11298a0, State: Running, Role: LEADER
I20260811 05:18:03.816080 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:18:03.816273 10228 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0610d500f32c4b83abc4e635c11298a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0610d500f32c4b83abc4e635c11298a0" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 44517 } }
I20260811 05:18:03.816962 10220 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0610d500f32c4b83abc4e635c11298a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:18:03.817554 10228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0610d500f32c4b83abc4e635c11298a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0610d500f32c4b83abc4e635c11298a0. Latest consensus state: current_term: 1 leader_uuid: "0610d500f32c4b83abc4e635c11298a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0610d500f32c4b83abc4e635c11298a0" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 44517 } } }
I20260811 05:18:03.817682 10228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0610d500f32c4b83abc4e635c11298a0 [sys.catalog]: This master's current role is: LEADER
I20260811 05:18:03.818123 10230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0610d500f32c4b83abc4e635c11298a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0610d500f32c4b83abc4e635c11298a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0610d500f32c4b83abc4e635c11298a0" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 44517 } } }
I20260811 05:18:03.818274 10240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:18:03.818276 10230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0610d500f32c4b83abc4e635c11298a0 [sys.catalog]: This master's current role is: LEADER
I20260811 05:18:03.821630 10240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:18:03.827955 10223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:03.818513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38547 (local address 127.7.67.190:44517)
0811 05:18:03.818629 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:03.818634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:03.818911 (+ 277us) server_negotiation.cc:407] Connection header received
0811 05:18:03.819138 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:03.819142 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:03.819193 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:03.819248 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:03.820273 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:03.821074 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:03.822233 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:03.822552 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:03.822671 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:03.824317 (+ 1646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:03.824321 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:03.824323 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:03.824348 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:03.826173 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:03.826623 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:03.826626 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:03.826628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:03.826718 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:03.827054 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:03.827057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:03.827059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:03.827236 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:03.827373 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:03.827619 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 05:18:03.827734 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 05:18:03.828933 10240 catalog_manager.cc:1347] Generated new cluster ID: 04489c01463f4b18afc55de9134d1ce4
I20260811 05:18:03.829041 10240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:18:03.839823 10240 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:18:03.841008 10240 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:18:03.865506 10240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0610d500f32c4b83abc4e635c11298a0: Generated new TSK 0
I20260811 05:18:03.866319 10240 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:18:03.874012 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 10121
2026-08-11T05:18:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1691 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:18:03 dist-test-slave-04bb krb5kdc[10266](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:18:03 dist-test-slave-04bb krb5kdc[10266](info): set up 2 sockets
Aug 11 05:18:03 dist-test-slave-04bb krb5kdc[10266](info): commencing operation
krb5kdc: starting...
W20260811 05:18:05.133793 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:18:05 dist-test-slave-04bb krb5kdc[10266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425485, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:18:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:18:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:18:05.263774 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:46053
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:42889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:46053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:18:05.377524 10290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:18:05.377835 10290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:18:05.377912 10290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:18:05.382246 10290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:18:05.382349 10290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:18:05.382395 10290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:18:05.382457 10290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:18:05.382501 10290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:18:05.388084 10290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:42889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:46053
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:46053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:18:05.389287 10290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:18:05.390367 10290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:18:05.400031 10300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:18:05.400133 10290 server_base.cc:1034] running on GCE node
W20260811 05:18:05.400068 10299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:18:05.400059 10302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:18:05.400919 10290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:18:05.402034 10290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:18:05.403210 10290 hybrid_clock.cc:648] HybridClock initialized: now 1786425485403185 us; error 35 us; skew 500 ppm
Aug 11 05:18:05 dist-test-slave-04bb krb5kdc[10266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425485, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:18:05.406306 10290 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:18:05.407366 10290 webserver.cc:460] Webserver started at http://127.7.67.190:44211/ using document root <none> and password file <none>
I20260811 05:18:05.407931 10290 fs_manager.cc:359] Metadata directory not provided
I20260811 05:18:05.407998 10290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:18:05.408236 10290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:18:05.410588 10290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "eefb23c6931f465ba2e25ae0d526b781"
format_stamp: "Formatted at 2026-08-11 05:18:05 on dist-test-slave-04bb"
server_key: "55f10d326184f74e2f4ac58c681972d5"
server_key_iv: "197183fdb8f753ea18acdc49bf8575cc"
server_key_version: "encryptionkey@0"
I20260811 05:18:05.411170 10290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "eefb23c6931f465ba2e25ae0d526b781"
format_stamp: "Formatted at 2026-08-11 05:18:05 on dist-test-slave-04bb"
server_key: "55f10d326184f74e2f4ac58c681972d5"
server_key_iv: "197183fdb8f753ea18acdc49bf8575cc"
server_key_version: "encryptionkey@0"
I20260811 05:18:05.415328 10290 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 05:18:05.417959 10311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:18:05.419338 10290 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 05:18:05.419497 10290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "eefb23c6931f465ba2e25ae0d526b781"
format_stamp: "Formatted at 2026-08-11 05:18:05 on dist-test-slave-04bb"
server_key: "55f10d326184f74e2f4ac58c681972d5"
server_key_iv: "197183fdb8f753ea18acdc49bf8575cc"
server_key_version: "encryptionkey@0"
I20260811 05:18:05.419602 10290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:18:05.439380 10290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:18:05.442922 10290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:18:05.443150 10290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:18:05.450811 10290 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:46053
I20260811 05:18:05.450845 10390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:46053 every 8 connection(s)
I20260811 05:18:05.451807 10290 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:18:05.454730 10391 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:18:05.460382 10391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eefb23c6931f465ba2e25ae0d526b781: Bootstrap starting.
I20260811 05:18:05.460758 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 10290
I20260811 05:18:05.460896 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:18:05.461192 7438 external_mini_cluster.cc:1428] Setting key 7fdb27184baedd640560efa6423358ff
I20260811 05:18:05.463183 10391 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eefb23c6931f465ba2e25ae0d526b781: Neither blocks nor log segments found. Creating new log.
I20260811 05:18:05.463924 10391 log.cc:824] T 00000000000000000000000000000000 P eefb23c6931f465ba2e25ae0d526b781: Log is configured to *not* fsync() on all Append() calls
I20260811 05:18:05.466017 10391 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eefb23c6931f465ba2e25ae0d526b781: No bootstrap required, opened a new log
I20260811 05:18:05.470237 10391 raft_consensus.cc:348] T 00000000000000000000000000000000 P eefb23c6931f465ba2e25ae0d526b781 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eefb23c6931f465ba2e25ae0d526b781" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 46053 } }
I20260811 05:18:05.470397 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:05.462541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55627 (local address 127.7.67.190:46053)
0811 05:18:05.462963 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:05.462981 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:05.463017 (+ 36us) server_negotiation.cc:407] Connection header received
0811 05:18:05.463715 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:05.463736 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:05.464133 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:05.464504 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:05.464946 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:05.465922 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:05.467117 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:05.467582 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:05.467704 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:05.468653 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:05.468658 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:05.468671 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:18:05.469066 (+ 395us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:05.469121 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:18:05.469146 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:05.469227 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:05.469408 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 05:18:05.469647 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":142,"threads_started":1}
I20260811 05:18:05.470453 10391 raft_consensus.cc:374] T 00000000000000000000000000000000 P eefb23c6931f465ba2e25ae0d526b781 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:18:05.470515 10391 raft_consensus.cc:729] T 00000000000000000000000000000000 P eefb23c6931f465ba2e25ae0d526b781 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eefb23c6931f465ba2e25ae0d526b781, State: Initialized, Role: FOLLOWER
I20260811 05:18:05.470996 10391 consensus_queue.cc:260] T 00000000000000000000000000000000 P eefb23c6931f465ba2e25ae0d526b781 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eefb23c6931f465ba2e25ae0d526b781" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 46053 } }
I20260811 05:18:05.471129 10391 raft_consensus.cc:388] T 00000000000000000000000000000000 P eefb23c6931f465ba2e25ae0d526b781 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:18:05.471215 10391 raft_consensus.cc:482] T 00000000000000000000000000000000 P eefb23c6931f465ba2e25ae0d526b781 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:18:05.471351 10391 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eefb23c6931f465ba2e25ae0d526b781 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:18:05.472250 10391 raft_consensus.cc:504] T 00000000000000000000000000000000 P eefb23c6931f465ba2e25ae0d526b781 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eefb23c6931f465ba2e25ae0d526b781" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 46053 } }
I20260811 05:18:05.472656 10391 leader_election.cc:302] T 00000000000000000000000000000000 P eefb23c6931f465ba2e25ae0d526b781 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eefb23c6931f465ba2e25ae0d526b781; no voters:
I20260811 05:18:05.472978 10391 leader_election.cc:288] T 00000000000000000000000000000000 P eefb23c6931f465ba2e25ae0d526b781 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:18:05.473105 10398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eefb23c6931f465ba2e25ae0d526b781 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:18:05.473390 10398 raft_consensus.cc:686] T 00000000000000000000000000000000 P eefb23c6931f465ba2e25ae0d526b781 [term 1 LEADER]: Becoming Leader. State: Replica: eefb23c6931f465ba2e25ae0d526b781, State: Running, Role: LEADER
I20260811 05:18:05.473412 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:18:05.473759 10398 consensus_queue.cc:237] T 00000000000000000000000000000000 P eefb23c6931f465ba2e25ae0d526b781 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eefb23c6931f465ba2e25ae0d526b781" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 46053 } }
I20260811 05:18:05.474342 10391 sys_catalog.cc:564] T 00000000000000000000000000000000 P eefb23c6931f465ba2e25ae0d526b781 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:18:05.475449 10401 sys_catalog.cc:455] T 00000000000000000000000000000000 P eefb23c6931f465ba2e25ae0d526b781 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eefb23c6931f465ba2e25ae0d526b781. Latest consensus state: current_term: 1 leader_uuid: "eefb23c6931f465ba2e25ae0d526b781" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eefb23c6931f465ba2e25ae0d526b781" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 46053 } } }
I20260811 05:18:05.475585 10401 sys_catalog.cc:458] T 00000000000000000000000000000000 P eefb23c6931f465ba2e25ae0d526b781 [sys.catalog]: This master's current role is: LEADER
I20260811 05:18:05.475925 10400 sys_catalog.cc:455] T 00000000000000000000000000000000 P eefb23c6931f465ba2e25ae0d526b781 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eefb23c6931f465ba2e25ae0d526b781" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eefb23c6931f465ba2e25ae0d526b781" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 46053 } } }
I20260811 05:18:05.476011 10400 sys_catalog.cc:458] T 00000000000000000000000000000000 P eefb23c6931f465ba2e25ae0d526b781 [sys.catalog]: This master's current role is: LEADER
W20260811 05:18:05.479648 10427 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eefb23c6931f465ba2e25ae0d526b781: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:18:05.479754 10427 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:18:05.479864 10417 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:18:05.480614 10417 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:18:05.482025 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:05.476678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45439 (local address 127.7.67.190:46053)
0811 05:18:05.476785 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:05.476789 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:05.477331 (+ 542us) server_negotiation.cc:407] Connection header received
0811 05:18:05.477478 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:05.477482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:05.477524 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:05.477619 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:05.477898 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:05.478620 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:05.479744 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:05.480066 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:05.480176 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:05.481256 (+ 1080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:05.481260 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:05.481263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:18:05.481276 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:05.481305 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:18:05.481320 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:05.481404 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:05.481830 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 05:18:05.481887 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 05:18:05.486455 10417 catalog_manager.cc:1347] Generated new cluster ID: e945cd958fc84ddebe5f31cb5082cbfd
I20260811 05:18:05.486541 10417 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:18:05.510324 10417 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:18:05.511636 10417 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:18:05.522094 10417 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eefb23c6931f465ba2e25ae0d526b781: Generated new TSK 0
I20260811 05:18:05.522964 10417 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:18:05.534101 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 10290
2026-08-11T05:18:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1667 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:18:05 dist-test-slave-04bb krb5kdc[10437](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:18:05 dist-test-slave-04bb krb5kdc[10437](info): set up 2 sockets
Aug 11 05:18:05 dist-test-slave-04bb krb5kdc[10437](info): commencing operation
krb5kdc: starting...
W20260811 05:18:06.820182 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:18:06 dist-test-slave-04bb krb5kdc[10437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425486, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:18:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:18:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:18:06.952085 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:37751
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:45549
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:37751
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:18:07.068195 10465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:18:07.068521 10465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:18:07.068627 10465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:18:07.072814 10465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:18:07.072907 10465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:18:07.072947 10465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:18:07.073006 10465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:18:07.073046 10465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:18:07.078323 10465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:45549
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:37751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:37751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10465
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:18:07.079440 10465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:18:07.080566 10465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:18:07.089510 10471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:18:07.089481 10475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:18:07.089470 10470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:18:07.089944 10465 server_base.cc:1034] running on GCE node
I20260811 05:18:07.090474 10465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:18:07.091562 10465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:18:07.092828 10465 hybrid_clock.cc:648] HybridClock initialized: now 1786425487092813 us; error 26 us; skew 500 ppm
Aug 11 05:18:07 dist-test-slave-04bb krb5kdc[10437](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425487, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:18:07.095711 10465 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:18:07.096652 10465 webserver.cc:460] Webserver started at http://127.7.67.190:40953/ using document root <none> and password file <none>
I20260811 05:18:07.097203 10465 fs_manager.cc:359] Metadata directory not provided
I20260811 05:18:07.097263 10465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:18:07.097435 10465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:18:07.099516 10465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "86c4d26b1cdc4adc845cfe7b769f4e89"
format_stamp: "Formatted at 2026-08-11 05:18:07 on dist-test-slave-04bb"
server_key: "af3cb8469a8f142a05ecb9e09b1644f5"
server_key_iv: "7bb2d880778fd15abf930e6dd2c27c36"
server_key_version: "encryptionkey@0"
I20260811 05:18:07.100054 10465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "86c4d26b1cdc4adc845cfe7b769f4e89"
format_stamp: "Formatted at 2026-08-11 05:18:07 on dist-test-slave-04bb"
server_key: "af3cb8469a8f142a05ecb9e09b1644f5"
server_key_iv: "7bb2d880778fd15abf930e6dd2c27c36"
server_key_version: "encryptionkey@0"
I20260811 05:18:07.103837 10465 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 05:18:07.106297 10485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:18:07.107524 10465 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:18:07.107671 10465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "86c4d26b1cdc4adc845cfe7b769f4e89"
format_stamp: "Formatted at 2026-08-11 05:18:07 on dist-test-slave-04bb"
server_key: "af3cb8469a8f142a05ecb9e09b1644f5"
server_key_iv: "7bb2d880778fd15abf930e6dd2c27c36"
server_key_version: "encryptionkey@0"
I20260811 05:18:07.107779 10465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:18:07.112447 10465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:18:07.115814 10465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:18:07.116029 10465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:18:07.123390 10465 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:37751
I20260811 05:18:07.123412 10560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:37751 every 8 connection(s)
I20260811 05:18:07.124305 10465 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:18:07.127161 10562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:18:07.128887 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 10465
I20260811 05:18:07.129031 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:18:07.129340 7438 external_mini_cluster.cc:1428] Setting key 8516926cb0a53e002fc693cab13c6edf
I20260811 05:18:07.133908 10562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86c4d26b1cdc4adc845cfe7b769f4e89: Bootstrap starting.
I20260811 05:18:07.134585 10568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:07.130777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60577 (local address 127.7.67.190:37751)
0811 05:18:07.131218 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:07.131233 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:07.131271 (+ 38us) server_negotiation.cc:407] Connection header received
0811 05:18:07.131894 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:07.131911 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:07.132181 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:07.132528 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:07.133351 (+ 823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:07.133356 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:07.133369 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:18:07.133381 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:07.133435 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:18:07.133456 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:07.133616 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:07.133690 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 05:18:07.133847 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":125,"threads_started":1}
I20260811 05:18:07.136245 10562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86c4d26b1cdc4adc845cfe7b769f4e89: Neither blocks nor log segments found. Creating new log.
I20260811 05:18:07.137152 10562 log.cc:824] T 00000000000000000000000000000000 P 86c4d26b1cdc4adc845cfe7b769f4e89: Log is configured to *not* fsync() on all Append() calls
I20260811 05:18:07.137449 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:18:07.139249 10562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86c4d26b1cdc4adc845cfe7b769f4e89: No bootstrap required, opened a new log
I20260811 05:18:07.141292 10568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:07.139493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49381 (local address 127.7.67.190:37751)
0811 05:18:07.139607 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:07.139610 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:07.139877 (+ 267us) server_negotiation.cc:407] Connection header received
0811 05:18:07.139995 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:07.139997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:07.140023 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:07.140087 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:07.140883 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:07.140886 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:07.140888 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 05:18:07.140898 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:07.140922 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 05:18:07.140932 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:07.141000 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:07.141112 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 05:18:07.141144 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 05:18:07.143483 10562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86c4d26b1cdc4adc845cfe7b769f4e89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86c4d26b1cdc4adc845cfe7b769f4e89" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37751 } }
I20260811 05:18:07.143700 10562 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86c4d26b1cdc4adc845cfe7b769f4e89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 05:18:07.143782 10562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86c4d26b1cdc4adc845cfe7b769f4e89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86c4d26b1cdc4adc845cfe7b769f4e89, State: Initialized, Role: FOLLOWER
I20260811 05:18:07.144234 10562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86c4d26b1cdc4adc845cfe7b769f4e89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86c4d26b1cdc4adc845cfe7b769f4e89" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37751 } }
I20260811 05:18:07.144382 10562 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86c4d26b1cdc4adc845cfe7b769f4e89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 05:18:07.144451 10562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86c4d26b1cdc4adc845cfe7b769f4e89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 05:18:07.144577 10562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86c4d26b1cdc4adc845cfe7b769f4e89 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:18:07.145483 10562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86c4d26b1cdc4adc845cfe7b769f4e89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86c4d26b1cdc4adc845cfe7b769f4e89" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37751 } }
I20260811 05:18:07.145887 10562 leader_election.cc:302] T 00000000000000000000000000000000 P 86c4d26b1cdc4adc845cfe7b769f4e89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86c4d26b1cdc4adc845cfe7b769f4e89; no voters:
I20260811 05:18:07.146274 10562 leader_election.cc:288] T 00000000000000000000000000000000 P 86c4d26b1cdc4adc845cfe7b769f4e89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 05:18:07.146346 10577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86c4d26b1cdc4adc845cfe7b769f4e89 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:18:07.146608 10577 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86c4d26b1cdc4adc845cfe7b769f4e89 [term 1 LEADER]: Becoming Leader. State: Replica: 86c4d26b1cdc4adc845cfe7b769f4e89, State: Running, Role: LEADER
I20260811 05:18:07.146937 10577 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86c4d26b1cdc4adc845cfe7b769f4e89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86c4d26b1cdc4adc845cfe7b769f4e89" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37751 } }
I20260811 05:18:07.147657 10562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86c4d26b1cdc4adc845cfe7b769f4e89 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:18:07.148831 10578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86c4d26b1cdc4adc845cfe7b769f4e89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86c4d26b1cdc4adc845cfe7b769f4e89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86c4d26b1cdc4adc845cfe7b769f4e89" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37751 } } }
I20260811 05:18:07.148972 10578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86c4d26b1cdc4adc845cfe7b769f4e89 [sys.catalog]: This master's current role is: LEADER
I20260811 05:18:07.149436 10595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:18:07.149507 10580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86c4d26b1cdc4adc845cfe7b769f4e89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86c4d26b1cdc4adc845cfe7b769f4e89. Latest consensus state: current_term: 1 leader_uuid: "86c4d26b1cdc4adc845cfe7b769f4e89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86c4d26b1cdc4adc845cfe7b769f4e89" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37751 } } }
I20260811 05:18:07.149681 10580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86c4d26b1cdc4adc845cfe7b769f4e89 [sys.catalog]: This master's current role is: LEADER
I20260811 05:18:07.152762 10595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:18:07.158587 10595 catalog_manager.cc:1347] Generated new cluster ID: 1f455ded87be4518a24eb077d998d4d5
I20260811 05:18:07.158659 10595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:18:07.168430 10595 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:18:07.168778 10595 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:18:07.178022 10595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86c4d26b1cdc4adc845cfe7b769f4e89: Generated new TSK 0
I20260811 05:18:07.178788 10595 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 05:18:07.184486 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 10465
2026-08-11T05:18:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1642 ms)
[----------] 18 tests from AuthTokenIssuingTest (29977 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:18:07 dist-test-slave-04bb krb5kdc[10611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:18:07 dist-test-slave-04bb krb5kdc[10611](info): set up 2 sockets
Aug 11 05:18:07 dist-test-slave-04bb krb5kdc[10611](info): commencing operation
krb5kdc: starting...
W20260811 05:18:08.463726 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:18:08 dist-test-slave-04bb krb5kdc[10611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:18:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:18:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:18:08.593559 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:37225
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:34345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:37225,127.7.67.189:45783,127.7.67.188:42917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:18:08.710332 10633 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 05:18:08.710664 10633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:18:08.710736 10633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:18:08.710810 10633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:18:08.714843 10633 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:18:08.714936 10633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:18:08.714975 10633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:18:08.715042 10633 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:18:08.715081 10633 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:18:08.720242 10633 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:34345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:37225,127.7.67.189:45783,127.7.67.188:42917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:37225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:18:08.721302 10633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:18:08.722283 10633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:18:08.730989 10644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:18:08.731061 10633 server_base.cc:1034] running on GCE node
W20260811 05:18:08.731057 10643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:18:08.731002 10648 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:18:08.731921 10633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:18:08.733107 10633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:18:08.734308 10633 hybrid_clock.cc:648] HybridClock initialized: now 1786425488734282 us; error 45 us; skew 500 ppm
Aug 11 05:18:08 dist-test-slave-04bb krb5kdc[10611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425488, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:18:08.737262 10633 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:18:08.738162 10633 webserver.cc:460] Webserver started at http://127.7.67.190:33833/ using document root <none> and password file <none>
I20260811 05:18:08.738749 10633 fs_manager.cc:359] Metadata directory not provided
I20260811 05:18:08.738847 10633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:18:08.739089 10633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:18:08.741225 10633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "673923c5d7884e938f5d830ff5a83c07"
format_stamp: "Formatted at 2026-08-11 05:18:08 on dist-test-slave-04bb"
server_key: "59873a78be61261a15b79d92bfcec39d"
server_key_iv: "e81ca425384c9efea30d271f345cd796"
server_key_version: "encryptionkey@0"
I20260811 05:18:08.741799 10633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "673923c5d7884e938f5d830ff5a83c07"
format_stamp: "Formatted at 2026-08-11 05:18:08 on dist-test-slave-04bb"
server_key: "59873a78be61261a15b79d92bfcec39d"
server_key_iv: "e81ca425384c9efea30d271f345cd796"
server_key_version: "encryptionkey@0"
I20260811 05:18:08.745671 10633 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 05:18:08.748165 10662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:18:08.749473 10633 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 05:18:08.749614 10633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "673923c5d7884e938f5d830ff5a83c07"
format_stamp: "Formatted at 2026-08-11 05:18:08 on dist-test-slave-04bb"
server_key: "59873a78be61261a15b79d92bfcec39d"
server_key_iv: "e81ca425384c9efea30d271f345cd796"
server_key_version: "encryptionkey@0"
I20260811 05:18:08.749740 10633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:18:08.768254 10633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:18:08.771636 10633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:18:08.771849 10633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:18:08.779137 10633 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:37225
I20260811 05:18:08.779428 10743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:37225 every 8 connection(s)
I20260811 05:18:08.780156 10633 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:18:08.783046 10744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:18:08.787077 10744 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37225 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:18:08.790158 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 10633
I20260811 05:18:08.790303 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:18:08.790601 7438 external_mini_cluster.cc:1428] Setting key 73ad1052944b0c303f9db7b895e4e9b7
Aug 11 05:18:08 dist-test-slave-04bb krb5kdc[10611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425488, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.189@KRBTEST.COM" created.
I20260811 05:18:08.802717 10752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:08.790201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58060 (local address 127.7.67.190:37225)
0811 05:18:08.790582 (+ 381us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:08.790590 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:08.790626 (+ 36us) server_negotiation.cc:407] Connection header received
0811 05:18:08.791479 (+ 853us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:08.791484 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:08.791661 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:08.791813 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:08.793011 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:08.793876 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:08.794886 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:08.795265 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:08.795445 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:08.797898 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:08.797904 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:08.797913 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:08.797934 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:08.799691 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:08.800168 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:08.800178 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:08.800179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:08.800311 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:08.800721 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:08.800723 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:08.800724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:08.801112 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:08.801225 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:08.801618 (+ 393us) server_negotiation.cc:299] Negotiation successful
0811 05:18:08.801896 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":72,"server-negotiator.queue_time_us":276,"thread_start_us":118,"threads_started":1}
I20260811 05:18:08.802723 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:08.789456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37225 (local address 127.0.0.1:58060)
0811 05:18:08.789955 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:08.790018 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 05:18:08.791153 (+ 1135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:18:08.791871 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:18:08.791880 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:18:08.792393 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:18:08.792831 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:08.792858 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:08.794012 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:08.794018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:08.794751 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:08.794760 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:08.795432 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:08.795436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:08.795530 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:08.796171 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:18:08.796187 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:18:08.797737 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:18:08.799808 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:08.799817 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:08.799827 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:08.800047 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:08.800440 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:08.800443 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:08.800445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:08.800627 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:08.801221 (+ 594us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:18:08.801226 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:18:08.801356 (+ 130us) client_negotiation.cc:769] Sending connection context
0811 05:18:08.801495 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 05:18:08.801753 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":120,"threads_started":1}
I20260811 05:18:08.805593 10744 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:18:08.806394 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:08.806127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:45783 (local address 127.0.0.1:55094)
0811 05:18:08.806254 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:08.806326 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.189:45783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 05:18:08.806609 10665 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.67.189:45783: connect: Connection refused (error 111)
I20260811 05:18:08.807456 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:08.807236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:45783 (local address 127.0.0.1:55108)
0811 05:18:08.807346 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:08.807404 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.189:45783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 05:18:08.807631 10744 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.189:45783: Network error: Client connection negotiation failed: client connection to 127.7.67.189:45783: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.189.keytab.
Entry for principal kudu/127.7.67.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.189.keytab.
I20260811 05:18:08.819099 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.189:45783
--webserver_interface=127.7.67.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.189.keytab
--principal=kudu/127.7.67.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:34345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:37225,127.7.67.189:45783,127.7.67.188:42917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:18:08.856877 10744 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 } attempt: 1
I20260811 05:18:08.857851 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:08.857491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:45783 (local address 127.0.0.1:55124)
0811 05:18:08.857674 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:08.857754 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.189:45783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 05:18:08.858636 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:08.858404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:45783 (local address 127.0.0.1:55126)
0811 05:18:08.858518 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:08.858572 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.189:45783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 05:18:08.858776 10744 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.189:45783: Network error: Client connection negotiation failed: client connection to 127.7.67.189:45783: connect: Connection refused (error 111)
I20260811 05:18:08.926961 10744 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 } attempt: 2
I20260811 05:18:08.927920 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:08.927546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:45783 (local address 127.0.0.1:55128)
0811 05:18:08.927730 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:08.927804 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.189:45783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 05:18:08.928710 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:08.928461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:45783 (local address 127.0.0.1:55144)
0811 05:18:08.928591 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:08.928645 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.189:45783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 05:18:08.928858 10744 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.189:45783: Network error: Client connection negotiation failed: client connection to 127.7.67.189:45783: connect: Connection refused (error 111)
W20260811 05:18:08.936383 10763 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 05:18:08.936741 10763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:18:08.936800 10763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:18:08.936919 10763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:18:08.941085 10763 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:18:08.941138 10763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:18:08.941159 10763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:18:08.941200 10763 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:18:08.941231 10763 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:18:08.946547 10763 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:34345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:37225,127.7.67.189:45783,127.7.67.188:42917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.189.keytab
--principal=kudu/127.7.67.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.189:45783
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.67.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:18:08.947639 10763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:18:08.948796 10763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:18:08.957558 10772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:18:08.957564 10771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:18:08.957635 10763 server_base.cc:1034] running on GCE node
W20260811 05:18:08.957573 10774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:18:08.958393 10763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:18:08.959476 10763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:18:08.960709 10763 hybrid_clock.cc:648] HybridClock initialized: now 1786425488960694 us; error 26 us; skew 500 ppm
Aug 11 05:18:08 dist-test-slave-04bb krb5kdc[10611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425488, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:18:08.968209 10763 init.cc:377] Logged in from keytab as kudu/127.7.67.189@KRBTEST.COM (short username kudu)
I20260811 05:18:08.969178 10763 webserver.cc:460] Webserver started at http://127.7.67.189:43023/ using document root <none> and password file <none>
I20260811 05:18:08.969719 10763 fs_manager.cc:359] Metadata directory not provided
I20260811 05:18:08.969779 10763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:18:08.969949 10763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:18:08.971952 10763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-1/data/instance:
uuid: "6216c364b55b446eaa89fac9c57f7d3d"
format_stamp: "Formatted at 2026-08-11 05:18:08 on dist-test-slave-04bb"
server_key: "3b8a3feca25b7dc4f6d529ee2d30449c"
server_key_iv: "0c94ef0ced5635bfffd5156cff9641ef"
server_key_version: "encryptionkey@0"
I20260811 05:18:08.972479 10763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-1/wal/instance:
uuid: "6216c364b55b446eaa89fac9c57f7d3d"
format_stamp: "Formatted at 2026-08-11 05:18:08 on dist-test-slave-04bb"
server_key: "3b8a3feca25b7dc4f6d529ee2d30449c"
server_key_iv: "0c94ef0ced5635bfffd5156cff9641ef"
server_key_version: "encryptionkey@0"
I20260811 05:18:08.976508 10763 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 05:18:08.979120 10783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:18:08.980464 10763 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 05:18:08.980662 10763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-1/wal
uuid: "6216c364b55b446eaa89fac9c57f7d3d"
format_stamp: "Formatted at 2026-08-11 05:18:08 on dist-test-slave-04bb"
server_key: "3b8a3feca25b7dc4f6d529ee2d30449c"
server_key_iv: "0c94ef0ced5635bfffd5156cff9641ef"
server_key_version: "encryptionkey@0"
I20260811 05:18:08.980767 10763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:18:08.998546 10763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:18:09.001507 10763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:18:09.001667 10763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:18:09.009364 10763 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.189:45783
I20260811 05:18:09.009636 10873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.189:45783 every 8 connection(s)
I20260811 05:18:09.010385 10763 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-1/data/info.pb
I20260811 05:18:09.013335 10874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:18:09.016058 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 10763
I20260811 05:18:09.016182 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-1/wal/instance
I20260811 05:18:09.016489 7438 external_mini_cluster.cc:1428] Setting key 11a015c6887157eedcff03c4071a6eb6
I20260811 05:18:09.017351 10874 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37225 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:18:09.020011 10744 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 } attempt: 3
WARNING: no policy specified for kudu/127.7.67.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.188@KRBTEST.COM" created.
Aug 11 05:18:09 dist-test-slave-04bb krb5kdc[10611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425488, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for kudu/127.7.67.189@KRBTEST.COM
Aug 11 05:18:09 dist-test-slave-04bb krb5kdc[10611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425488, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.189@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:18:09.035107 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.020436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:45783 (local address 127.0.0.1:55154)
0811 05:18:09.020725 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:09.020739 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:18:09.020821 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:18:09.022029 (+ 1208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:18:09.022034 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:18:09.022257 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:18:09.022422 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:09.022433 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.023862 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.023866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:09.024709 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:09.024720 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.025657 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.025662 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:09.025790 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:09.026401 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:18:09.026418 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:18:09.030091 (+ 3673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:18:09.032890 (+ 2799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:09.032894 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:09.032897 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:09.033250 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:09.033689 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:09.033695 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:09.033698 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:09.033804 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:09.034496 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:18:09.034500 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:18:09.034617 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 05:18:09.034733 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 05:18:09.034894 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213}
I20260811 05:18:09.036561 10752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.019269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58076 (local address 127.7.67.190:37225)
0811 05:18:09.019414 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.019418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.020550 (+ 1132us) server_negotiation.cc:407] Connection header received
0811 05:18:09.021800 (+ 1250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.021804 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.021861 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.021919 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.023014 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.023926 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.025206 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.025455 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.025564 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:09.031897 (+ 6333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:09.031902 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:09.031904 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:09.031936 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:09.034526 (+ 2590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.035101 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.035104 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.035106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.035213 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.035544 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.035549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.035553 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.035782 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:09.035851 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:09.036266 (+ 415us) server_negotiation.cc:299] Negotiation successful
0811 05:18:09.036390 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 05:18:09.036634 10877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.019779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37225 (local address 127.0.0.1:58076)
0811 05:18:09.020381 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:09.020441 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 05:18:09.021567 (+ 1126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:18:09.022119 (+ 552us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:18:09.022123 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:18:09.022558 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:18:09.022841 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:09.022862 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.024067 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.024071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:09.025081 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:09.025092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.025615 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.025620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:09.025770 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:09.026529 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:18:09.026556 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:18:09.031743 (+ 5187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:18:09.034633 (+ 2890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:09.034640 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:09.034646 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:09.034979 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:09.035340 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:09.035343 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:09.035345 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:09.035433 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:09.035921 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:18:09.035926 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:18:09.036030 (+ 104us) client_negotiation.cc:769] Sending connection context
0811 05:18:09.036175 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 05:18:09.036381 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"mutex_wait_us":50,"thread_start_us":173,"threads_started":1}
I20260811 05:18:09.036641 10879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.020843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55154 (local address 127.7.67.189:45783)
0811 05:18:09.021173 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.021184 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.021227 (+ 43us) server_negotiation.cc:407] Connection header received
0811 05:18:09.021567 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.021593 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.021829 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.022056 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.022556 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.023736 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.024847 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.025486 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.026062 (+ 576us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:09.030268 (+ 4206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:09.030279 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:09.030298 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:09.030332 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:09.032743 (+ 2411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.033413 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.033423 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.033433 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.033589 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.033927 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.033930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.033932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.034377 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:09.034534 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:09.034967 (+ 433us) server_negotiation.cc:299] Negotiation successful
0811 05:18:09.035312 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":140,"threads_started":1}
I20260811 05:18:09.038905 10744 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:18:09.039065 10874 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:18:09.039662 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.039420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:42917 (local address 127.0.0.1:56732)
0811 05:18:09.039548 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:09.039599 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 05:18:09.040514 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.040254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:42917 (local address 127.0.0.1:56738)
0811 05:18:09.040415 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:09.040464 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260811 05:18:09.040727 10744 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.188:42917: Network error: Client connection negotiation failed: client connection to 127.7.67.188:42917: connect: Connection refused (error 111)
Aug 11 05:18:09 dist-test-slave-04bb krb5kdc[10611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425488, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.189@KRBTEST.COM for kudu/127.7.67.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.188.keytab.
Entry for principal kudu/127.7.67.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.188.keytab.
I20260811 05:18:09.047374 10877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.039731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:45783 (local address 127.0.0.1:55166)
0811 05:18:09.039849 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:09.039865 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:18:09.039952 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:18:09.040247 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:18:09.040249 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:18:09.040388 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:18:09.040460 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:09.040465 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.041422 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.041425 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:09.041950 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:09.041958 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.042450 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.042453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:09.042526 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:09.042979 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:18:09.042991 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:18:09.044240 (+ 1249us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:18:09.045848 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:09.045851 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:09.045852 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:09.046128 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:09.046410 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:09.046416 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:09.046420 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:09.046533 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:09.046890 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:18:09.046893 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:18:09.047018 (+ 125us) client_negotiation.cc:769] Sending connection context
0811 05:18:09.047123 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 05:18:09.047263 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 05:18:09.047580 10879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.039855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55166 (local address 127.7.67.189:45783)
0811 05:18:09.039961 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.039966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.039982 (+ 16us) server_negotiation.cc:407] Connection header received
0811 05:18:09.040019 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.040022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.040069 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.040153 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.040595 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.041305 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.042109 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.042343 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.042428 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:09.044372 (+ 1944us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:09.044376 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:09.044377 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:09.044398 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:09.045746 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.046217 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.046220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.046221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.046292 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.046618 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.046621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.046622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.046781 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:09.046868 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:09.047251 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 05:18:09.047391 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 05:18:09.048138 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.188:42917
--webserver_interface=127.7.67.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.188.keytab
--principal=kudu/127.7.67.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:34345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:37225,127.7.67.189:45783,127.7.67.188:42917
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:18:09.048326 10874 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:18:09.049140 10877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.048807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:42917 (local address 127.0.0.1:56752)
0811 05:18:09.048961 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:09.049058 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 05:18:09.049402 10785 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.67.188:42917: connect: Connection refused (error 111)
I20260811 05:18:09.050387 10877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.050108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:42917 (local address 127.0.0.1:56764)
0811 05:18:09.050242 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:09.050304 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 05:18:09.050546 10874 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.188:42917: Network error: Client connection negotiation failed: client connection to 127.7.67.188:42917: connect: Connection refused (error 111)
I20260811 05:18:09.071920 10744 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } attempt: 1
I20260811 05:18:09.072858 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.072521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:42917 (local address 127.0.0.1:56772)
0811 05:18:09.072712 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:09.072775 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 05:18:09.073707 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.073475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:42917 (local address 127.0.0.1:56784)
0811 05:18:09.073584 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:09.073642 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 05:18:09.073838 10744 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.188:42917: Network error: Client connection negotiation failed: client connection to 127.7.67.188:42917: connect: Connection refused (error 111)
I20260811 05:18:09.099828 10874 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } attempt: 1
I20260811 05:18:09.100807 10877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.100463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:42917 (local address 127.0.0.1:56798)
0811 05:18:09.100664 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:09.100726 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 05:18:09.101603 10877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.101381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:42917 (local address 127.0.0.1:56808)
0811 05:18:09.101496 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:09.101549 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 05:18:09.101732 10874 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.188:42917: Network error: Client connection negotiation failed: client connection to 127.7.67.188:42917: connect: Connection refused (error 111)
I20260811 05:18:09.148999 10744 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } attempt: 2
I20260811 05:18:09.149935 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.149619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:42917 (local address 127.0.0.1:56810)
0811 05:18:09.149786 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:09.149852 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 05:18:09.150684 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.150468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:42917 (local address 127.0.0.1:56812)
0811 05:18:09.150578 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:09.150628 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 05:18:09.150820 10744 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.188:42917: Network error: Client connection negotiation failed: client connection to 127.7.67.188:42917: connect: Connection refused (error 111)
W20260811 05:18:09.166215 10881 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 05:18:09.166524 10881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:18:09.166591 10881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:18:09.166674 10881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 05:18:09.169881 10874 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } attempt: 2
I20260811 05:18:09.170647 10877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.170356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:42917 (local address 127.0.0.1:56816)
0811 05:18:09.170506 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:09.170573 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 05:18:09.170743 10881 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:18:09.170836 10881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:18:09.170908 10881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:18:09.170948 10881 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:18:09.170984 10881 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:18:09.171420 10877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.171210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:42917 (local address 127.0.0.1:56818)
0811 05:18:09.171317 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:09.171364 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 05:18:09.171552 10874 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.188:42917: Network error: Client connection negotiation failed: client connection to 127.7.67.188:42917: connect: Connection refused (error 111)
I20260811 05:18:09.176406 10881 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:34345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:37225,127.7.67.189:45783,127.7.67.188:42917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.188.keytab
--principal=kudu/127.7.67.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.188:42917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.67.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:18:09.177474 10881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:18:09.178516 10881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:18:09.187307 10892 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:18:09.187335 10888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:18:09.187340 10890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:18:09.187625 10881 server_base.cc:1034] running on GCE node
I20260811 05:18:09.188175 10881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:18:09.189349 10881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:18:09.190550 10881 hybrid_clock.cc:648] HybridClock initialized: now 1786425489190534 us; error 26 us; skew 500 ppm
Aug 11 05:18:09 dist-test-slave-04bb krb5kdc[10611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425489, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:18:09.193430 10881 init.cc:377] Logged in from keytab as kudu/127.7.67.188@KRBTEST.COM (short username kudu)
I20260811 05:18:09.194391 10881 webserver.cc:460] Webserver started at http://127.7.67.188:45243/ using document root <none> and password file <none>
I20260811 05:18:09.194973 10881 fs_manager.cc:359] Metadata directory not provided
I20260811 05:18:09.195067 10881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:18:09.195308 10881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:18:09.197501 10881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-2/data/instance:
uuid: "d92df9b3deea4f8381e98c50c78938dd"
format_stamp: "Formatted at 2026-08-11 05:18:09 on dist-test-slave-04bb"
server_key: "54e6ae61a9778c8064af7ad2605f4f44"
server_key_iv: "b47d4f44b7a96b9a849c9c62e9b8b53f"
server_key_version: "encryptionkey@0"
I20260811 05:18:09.198076 10881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-2/wal/instance:
uuid: "d92df9b3deea4f8381e98c50c78938dd"
format_stamp: "Formatted at 2026-08-11 05:18:09 on dist-test-slave-04bb"
server_key: "54e6ae61a9778c8064af7ad2605f4f44"
server_key_iv: "b47d4f44b7a96b9a849c9c62e9b8b53f"
server_key_version: "encryptionkey@0"
I20260811 05:18:09.201870 10881 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 05:18:09.204221 10905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:18:09.205489 10881 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:18:09.205648 10881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-2/wal
uuid: "d92df9b3deea4f8381e98c50c78938dd"
format_stamp: "Formatted at 2026-08-11 05:18:09 on dist-test-slave-04bb"
server_key: "54e6ae61a9778c8064af7ad2605f4f44"
server_key_iv: "b47d4f44b7a96b9a849c9c62e9b8b53f"
server_key_version: "encryptionkey@0"
I20260811 05:18:09.205765 10881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:18:09.242722 10881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:18:09.246304 10881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:18:09.246527 10881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:18:09.249979 10744 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } attempt: 3
I20260811 05:18:09.250895 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.250497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:42917 (local address 127.0.0.1:56820)
0811 05:18:09.250727 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:09.250797 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
I20260811 05:18:09.251778 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.251560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:42917 (local address 127.0.0.1:56828)
0811 05:18:09.251693 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:09.251730 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:42917: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
W20260811 05:18:09.251947 10744 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.188:42917: Network error: Client connection negotiation failed: client connection to 127.7.67.188:42917: connect: Connection refused (error 111)
I20260811 05:18:09.254716 10881 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.188:42917
I20260811 05:18:09.254937 10981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.188:42917 every 8 connection(s)
I20260811 05:18:09.255746 10881 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-2/data/info.pb
I20260811 05:18:09.256661 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 10881
I20260811 05:18:09.256790 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786425190664461-7438-0/minicluster-data/master-2/wal/instance
I20260811 05:18:09.257108 7438 external_mini_cluster.cc:1428] Setting key 7ecc844b835da6aa4e8550f84a75656e
I20260811 05:18:09.259286 10982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:18:09.262724 10874 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } attempt: 3
I20260811 05:18:09.263310 10982 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37225 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:18:09 dist-test-slave-04bb krb5kdc[10611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.188@KRBTEST.COM
Aug 11 05:18:09 dist-test-slave-04bb krb5kdc[10611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425488, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.189@KRBTEST.COM for kudu/127.7.67.188@KRBTEST.COM
Aug 11 05:18:09 dist-test-slave-04bb krb5kdc[10611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425489, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.188@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:18:09.273082 10877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.263189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:42917 (local address 127.0.0.1:56846)
0811 05:18:09.263342 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:09.263358 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:18:09.263450 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:18:09.263987 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:18:09.263991 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:18:09.264184 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:18:09.264294 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:09.264301 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.265349 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.265362 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:09.266144 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:09.266159 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.266759 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.266764 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:09.266862 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:09.267308 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:18:09.267324 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:18:09.268841 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:18:09.270998 (+ 2157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:09.271003 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:09.271007 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:09.271287 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:09.271720 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:09.271723 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:09.271726 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:09.271825 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:09.272509 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:18:09.272511 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:18:09.272649 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 05:18:09.272763 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 05:18:09.272917 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 05:18:09.274448 10994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.263391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56846 (local address 127.7.67.188:42917)
0811 05:18:09.263741 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.263749 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.263761 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:18:09.263792 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.263794 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.263834 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.263935 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.264440 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.265196 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.266282 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.266616 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.266818 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:09.268979 (+ 2161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:09.268983 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:09.268986 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:09.269010 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:09.270874 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.271438 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.271447 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.271454 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.271600 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.271930 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.271933 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.271934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.272303 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:09.272465 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:09.272936 (+ 471us) server_negotiation.cc:299] Negotiation successful
0811 05:18:09.273622 (+ 686us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":284,"spinlock_wait_cycles":10880,"thread_start_us":184,"threads_started":1}
I20260811 05:18:09.275178 10993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.258533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56842 (local address 127.7.67.188:42917)
0811 05:18:09.258984 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.259000 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.259038 (+ 38us) server_negotiation.cc:407] Connection header received
0811 05:18:09.259803 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.259823 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.260199 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.260589 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.261437 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.262431 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.263546 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.264233 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.264419 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:09.268432 (+ 4013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:09.268441 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:09.268455 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:09.268489 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:09.271719 (+ 3230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.272229 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.272232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.272234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.272336 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.272710 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.272715 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.272716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.272933 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:09.273089 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:09.273482 (+ 393us) server_negotiation.cc:299] Negotiation successful
0811 05:18:09.274005 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"spinlock_wait_cycles":896,"thread_start_us":138,"threads_started":1}
I20260811 05:18:09.276703 10752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.265416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58086 (local address 127.7.67.190:37225)
0811 05:18:09.265547 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.265553 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.265826 (+ 273us) server_negotiation.cc:407] Connection header received
0811 05:18:09.266005 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.266008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.266063 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.266147 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.267310 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.267900 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.269447 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.269813 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.269903 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:09.272992 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:09.272996 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:09.272998 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:09.273025 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:09.274612 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.275088 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.275092 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.275093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.275187 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.275639 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.275642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.275643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.275832 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:09.275955 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:09.276324 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 05:18:09.276475 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 05:18:09.277448 10995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.265355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:37225 (local address 127.0.0.1:58086)
0811 05:18:09.265709 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:09.265749 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 05:18:09.265895 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:18:09.266226 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:18:09.266235 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:18:09.266688 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:18:09.266943 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:09.266953 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.268042 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.268048 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:09.269287 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:09.269302 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.270081 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.270087 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:09.270229 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:09.270749 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:18:09.270760 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:18:09.272796 (+ 2036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:18:09.274737 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:09.274746 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:09.274751 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:09.274984 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:09.275306 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:09.275309 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:09.275311 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:09.275416 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:09.275956 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:18:09.275964 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:18:09.276074 (+ 110us) client_negotiation.cc:769] Sending connection context
0811 05:18:09.276860 (+ 786us) client_negotiation.cc:240] Negotiation successful
0811 05:18:09.277237 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":27,"thread_start_us":86,"threads_started":1}
I20260811 05:18:09.278743 10982 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:18:09.280112 10874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6216c364b55b446eaa89fac9c57f7d3d: Bootstrap starting.
Aug 11 05:18:09 dist-test-slave-04bb krb5kdc[10611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:18:09.282713 10874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6216c364b55b446eaa89fac9c57f7d3d: Neither blocks nor log segments found. Creating new log.
I20260811 05:18:09.283630 10874 log.cc:824] T 00000000000000000000000000000000 P 6216c364b55b446eaa89fac9c57f7d3d: Log is configured to *not* fsync() on all Append() calls
Aug 11 05:18:09 dist-test-slave-04bb krb5kdc[10611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425489, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.188@KRBTEST.COM for kudu/127.7.67.189@KRBTEST.COM
I20260811 05:18:09.286110 10874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6216c364b55b446eaa89fac9c57f7d3d: No bootstrap required, opened a new log
I20260811 05:18:09.287117 10752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.277136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58092 (local address 127.7.67.190:37225)
0811 05:18:09.277283 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.277289 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.277303 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:18:09.277347 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.277350 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.277417 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.277507 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.278252 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.278828 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.279907 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.280133 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.280224 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:09.282904 (+ 2680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:09.282908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:09.282911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:09.282934 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:09.284939 (+ 2005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.285496 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.285499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.285501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.285589 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.285928 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.285933 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.285936 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.286158 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:09.286401 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:09.286805 (+ 404us) server_negotiation.cc:299] Negotiation successful
0811 05:18:09.286929 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 05:18:09.289009 10995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.279195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:45783 (local address 127.0.0.1:55180)
0811 05:18:09.279322 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:09.279338 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 05:18:09.279423 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:18:09.279836 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:18:09.279839 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:18:09.279998 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:18:09.280104 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:09.280111 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.281473 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.281477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:09.282206 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:09.282218 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.282883 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.282888 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:09.283005 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:09.283533 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:18:09.283547 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:18:09.284890 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:18:09.287065 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:09.287068 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:09.287070 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:09.287407 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:09.287741 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:09.287744 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:09.287746 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:09.287835 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:09.288349 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:18:09.288351 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:18:09.288585 (+ 234us) client_negotiation.cc:769] Sending connection context
0811 05:18:09.288702 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 05:18:09.288827 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 05:18:09.289041 10879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.279429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55180 (local address 127.7.67.189:45783)
0811 05:18:09.279536 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.279540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.279552 (+ 12us) server_negotiation.cc:407] Connection header received
0811 05:18:09.279667 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.279670 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.279716 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.279803 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.280233 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.281297 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.282343 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.282742 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.282874 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:09.285029 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:09.285033 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:09.285036 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:09.285062 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:09.286949 (+ 1887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.287518 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.287525 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.287530 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.287628 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.287979 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.287983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.287985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.288227 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:09.288333 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:09.288777 (+ 444us) server_negotiation.cc:299] Negotiation successful
0811 05:18:09.288892 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 05:18:09.290006 10982 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:18:09.290354 10874 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6216c364b55b446eaa89fac9c57f7d3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673923c5d7884e938f5d830ff5a83c07" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37225 } } peers { permanent_uuid: "6216c364b55b446eaa89fac9c57f7d3d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 } } peers { permanent_uuid: "d92df9b3deea4f8381e98c50c78938dd" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } }
I20260811 05:18:09.290681 10874 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6216c364b55b446eaa89fac9c57f7d3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6216c364b55b446eaa89fac9c57f7d3d, State: Initialized, Role: FOLLOWER
I20260811 05:18:09.291054 10874 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6216c364b55b446eaa89fac9c57f7d3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673923c5d7884e938f5d830ff5a83c07" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37225 } } peers { permanent_uuid: "6216c364b55b446eaa89fac9c57f7d3d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 } } peers { permanent_uuid: "d92df9b3deea4f8381e98c50c78938dd" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } }
I20260811 05:18:09.291553 10997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6216c364b55b446eaa89fac9c57f7d3d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673923c5d7884e938f5d830ff5a83c07" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37225 } } peers { permanent_uuid: "6216c364b55b446eaa89fac9c57f7d3d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 } } peers { permanent_uuid: "d92df9b3deea4f8381e98c50c78938dd" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } } }
I20260811 05:18:09.291788 10997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6216c364b55b446eaa89fac9c57f7d3d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:18:09.292397 10874 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6216c364b55b446eaa89fac9c57f7d3d [sys.catalog]: configured and running, proceeding with master startup.
Aug 11 05:18:09 dist-test-slave-04bb krb5kdc[10611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425489, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.188@KRBTEST.COM for kudu/127.7.67.188@KRBTEST.COM
Aug 11 05:18:09 dist-test-slave-04bb krb5kdc[10611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425488, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.189@KRBTEST.COM
W20260811 05:18:09.298265 11018 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6216c364b55b446eaa89fac9c57f7d3d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:18:09.298405 11018 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:18:09.296380 10763 master_runner.cc:186] Error getting master registration for 127.7.67.190:37225: OK, instance_id { permanent_uuid: "673923c5d7884e938f5d830ff5a83c07" instance_seqno: 1786425488777219 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 05:18:09.300263 10995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.290491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:42917 (local address 127.0.0.1:56858)
0811 05:18:09.290625 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:09.290638 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 05:18:09.290714 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:18:09.290984 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:18:09.290986 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:18:09.291144 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:18:09.291212 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:09.291217 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.292173 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.292176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:09.292952 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:09.292964 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.293615 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.293620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:09.293723 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:09.294187 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:18:09.294206 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:18:09.296044 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:18:09.298457 (+ 2413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:09.298459 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:09.298461 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:09.298780 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:09.299117 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:09.299121 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:09.299125 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:09.299292 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:09.299794 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:18:09.299796 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:18:09.299883 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 05:18:09.299978 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 05:18:09.300109 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 05:18:09.300273 10879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.289290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55194 (local address 127.7.67.189:45783)
0811 05:18:09.289412 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.289416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.289635 (+ 219us) server_negotiation.cc:407] Connection header received
0811 05:18:09.289787 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.289790 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.289831 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.289911 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.290769 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.291478 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.292475 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.292817 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.292898 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:09.296623 (+ 3725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:09.296627 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:09.296629 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:09.296653 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:09.298303 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.298808 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.298811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.298813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.298912 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.299235 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.299238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.299240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.299473 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:09.299615 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:09.300012 (+ 397us) server_negotiation.cc:299] Negotiation successful
0811 05:18:09.300128 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 05:18:09.300755 10993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.290517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56858 (local address 127.7.67.188:42917)
0811 05:18:09.290612 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.290616 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.290679 (+ 63us) server_negotiation.cc:407] Connection header received
0811 05:18:09.290799 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.290802 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.290835 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.290878 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.291355 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.292047 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.293085 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.293478 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.293587 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:09.296161 (+ 2574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:09.296165 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:09.296167 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:09.296189 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:09.298339 (+ 2150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.298896 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.298898 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.298900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.299024 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.299398 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.299401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.299403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.299670 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:09.299763 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:09.300462 (+ 699us) server_negotiation.cc:299] Negotiation successful
0811 05:18:09.300617 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 05:18:09.300879 10763 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 05:18:09.304117 10982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d92df9b3deea4f8381e98c50c78938dd: Bootstrap starting.
I20260811 05:18:09.304416 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:18:09.305915 10706 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "673923c5d7884e938f5d830ff5a83c07"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58092
I20260811 05:18:09.307230 10982 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d92df9b3deea4f8381e98c50c78938dd: Neither blocks nor log segments found. Creating new log.
I20260811 05:18:09.308144 10982 log.cc:824] T 00000000000000000000000000000000 P d92df9b3deea4f8381e98c50c78938dd: Log is configured to *not* fsync() on all Append() calls
I20260811 05:18:09.310572 10982 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d92df9b3deea4f8381e98c50c78938dd: No bootstrap required, opened a new log
I20260811 05:18:09.314745 10982 raft_consensus.cc:348] T 00000000000000000000000000000000 P d92df9b3deea4f8381e98c50c78938dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673923c5d7884e938f5d830ff5a83c07" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37225 } } peers { permanent_uuid: "6216c364b55b446eaa89fac9c57f7d3d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 } } peers { permanent_uuid: "d92df9b3deea4f8381e98c50c78938dd" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } }
I20260811 05:18:09.315018 10982 raft_consensus.cc:729] T 00000000000000000000000000000000 P d92df9b3deea4f8381e98c50c78938dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d92df9b3deea4f8381e98c50c78938dd, State: Initialized, Role: FOLLOWER
I20260811 05:18:09.315374 10982 consensus_queue.cc:260] T 00000000000000000000000000000000 P d92df9b3deea4f8381e98c50c78938dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673923c5d7884e938f5d830ff5a83c07" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37225 } } peers { permanent_uuid: "6216c364b55b446eaa89fac9c57f7d3d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 } } peers { permanent_uuid: "d92df9b3deea4f8381e98c50c78938dd" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } }
I20260811 05:18:09.315995 11024 sys_catalog.cc:455] T 00000000000000000000000000000000 P d92df9b3deea4f8381e98c50c78938dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673923c5d7884e938f5d830ff5a83c07" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37225 } } peers { permanent_uuid: "6216c364b55b446eaa89fac9c57f7d3d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 } } peers { permanent_uuid: "d92df9b3deea4f8381e98c50c78938dd" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } } }
I20260811 05:18:09.316207 11024 sys_catalog.cc:458] T 00000000000000000000000000000000 P d92df9b3deea4f8381e98c50c78938dd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:18:09.316942 10982 sys_catalog.cc:564] T 00000000000000000000000000000000 P d92df9b3deea4f8381e98c50c78938dd [sys.catalog]: configured and running, proceeding with master startup.
W20260811 05:18:09.322165 11042 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d92df9b3deea4f8381e98c50c78938dd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:18:09.322250 11042 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:18:09.320740 10881 master_runner.cc:186] Error getting master registration for 127.7.67.190:37225: OK, instance_id { permanent_uuid: "673923c5d7884e938f5d830ff5a83c07" instance_seqno: 1786425488777219 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 05:18:09.323606 10881 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 05:18:09.416152 10744 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } attempt: 4
Aug 11 05:18:09 dist-test-slave-04bb krb5kdc[10611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425488, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for kudu/127.7.67.188@KRBTEST.COM
I20260811 05:18:09.427253 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.416879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:42917 (local address 127.0.0.1:56874)
0811 05:18:09.417074 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:09.417094 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 05:18:09.417219 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:18:09.417614 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:18:09.417617 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:18:09.417907 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:18:09.418069 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:09.418077 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.419224 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.419229 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:09.420151 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:09.420165 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.420859 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.420863 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:09.420950 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:09.421581 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:18:09.421597 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:18:09.423287 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:18:09.425485 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:09.425488 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:09.425491 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:09.425772 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:09.426086 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:09.426088 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:09.426091 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:09.426185 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:09.426639 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:18:09.426642 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:18:09.426769 (+ 127us) client_negotiation.cc:769] Sending connection context
0811 05:18:09.426908 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 05:18:09.427058 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 05:18:09.427301 10993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.417239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56874 (local address 127.7.67.188:42917)
0811 05:18:09.417366 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.417371 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.417387 (+ 16us) server_negotiation.cc:407] Connection header received
0811 05:18:09.417438 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.417440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.417493 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.417582 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.418231 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.419083 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.420307 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.420738 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.420852 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:09.423443 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:09.423447 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:09.423450 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:09.423479 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:09.425351 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.425882 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.425886 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.425888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.425986 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.426297 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.426300 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.426302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.426534 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:09.426644 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:09.426980 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 05:18:09.427112 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 05:18:09.431346 10744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07: Bootstrap starting.
I20260811 05:18:09.434309 10744 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07: Neither blocks nor log segments found. Creating new log.
I20260811 05:18:09.435211 10744 log.cc:824] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07: Log is configured to *not* fsync() on all Append() calls
I20260811 05:18:09.437659 10744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07: No bootstrap required, opened a new log
I20260811 05:18:09.441089 10744 raft_consensus.cc:348] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673923c5d7884e938f5d830ff5a83c07" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37225 } } peers { permanent_uuid: "6216c364b55b446eaa89fac9c57f7d3d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 } } peers { permanent_uuid: "d92df9b3deea4f8381e98c50c78938dd" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } }
I20260811 05:18:09.441349 10744 raft_consensus.cc:729] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 673923c5d7884e938f5d830ff5a83c07, State: Initialized, Role: FOLLOWER
I20260811 05:18:09.441704 10744 consensus_queue.cc:260] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673923c5d7884e938f5d830ff5a83c07" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37225 } } peers { permanent_uuid: "6216c364b55b446eaa89fac9c57f7d3d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 } } peers { permanent_uuid: "d92df9b3deea4f8381e98c50c78938dd" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } }
I20260811 05:18:09.442348 11049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673923c5d7884e938f5d830ff5a83c07" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37225 } } peers { permanent_uuid: "6216c364b55b446eaa89fac9c57f7d3d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 } } peers { permanent_uuid: "d92df9b3deea4f8381e98c50c78938dd" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } } }
I20260811 05:18:09.442669 11049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:18:09.443192 10744 sys_catalog.cc:564] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:18:09.443277 10706 raft_consensus.cc:482] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 05:18:09.443403 10706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:18:09.444905 10706 raft_consensus.cc:504] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673923c5d7884e938f5d830ff5a83c07" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37225 } } peers { permanent_uuid: "6216c364b55b446eaa89fac9c57f7d3d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 } } peers { permanent_uuid: "d92df9b3deea4f8381e98c50c78938dd" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } }
I20260811 05:18:09.445976 10706 leader_election.cc:288] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07 [CANDIDATE]: Term 1 election: Requested vote from peers 6216c364b55b446eaa89fac9c57f7d3d (127.7.67.189:45783), d92df9b3deea4f8381e98c50c78938dd (127.7.67.188:42917)
I20260811 05:18:09.446969 10821 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "673923c5d7884e938f5d830ff5a83c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6216c364b55b446eaa89fac9c57f7d3d"
I20260811 05:18:09.447263 10821 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6216c364b55b446eaa89fac9c57f7d3d [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:18:09.448617 10943 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "673923c5d7884e938f5d830ff5a83c07" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d92df9b3deea4f8381e98c50c78938dd"
I20260811 05:18:09.449236 10943 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d92df9b3deea4f8381e98c50c78938dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:18:09.450688 10943 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d92df9b3deea4f8381e98c50c78938dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 673923c5d7884e938f5d830ff5a83c07 in term 1.
I20260811 05:18:09.451750 10664 leader_election.cc:302] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 673923c5d7884e938f5d830ff5a83c07, d92df9b3deea4f8381e98c50c78938dd; no voters:
I20260811 05:18:09.452133 11049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 05:18:09.452428 11049 raft_consensus.cc:686] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07 [term 1 LEADER]: Becoming Leader. State: Replica: 673923c5d7884e938f5d830ff5a83c07, State: Running, Role: LEADER
I20260811 05:18:09.452919 11049 consensus_queue.cc:237] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673923c5d7884e938f5d830ff5a83c07" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37225 } } peers { permanent_uuid: "6216c364b55b446eaa89fac9c57f7d3d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 } } peers { permanent_uuid: "d92df9b3deea4f8381e98c50c78938dd" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } }
I20260811 05:18:09.453612 10821 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6216c364b55b446eaa89fac9c57f7d3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 673923c5d7884e938f5d830ff5a83c07 in term 1.
I20260811 05:18:09.454993 11049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 673923c5d7884e938f5d830ff5a83c07. Latest consensus state: current_term: 1 leader_uuid: "673923c5d7884e938f5d830ff5a83c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673923c5d7884e938f5d830ff5a83c07" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37225 } } peers { permanent_uuid: "6216c364b55b446eaa89fac9c57f7d3d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 } } peers { permanent_uuid: "d92df9b3deea4f8381e98c50c78938dd" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } } }
I20260811 05:18:09.455148 11049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07 [sys.catalog]: This master's current role is: LEADER
I20260811 05:18:09.458278 11073 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:18:09.459403 10633 master_runner.cc:263] No leader master found from master 673923c5d7884e938f5d830ff5a83c07
I20260811 05:18:09.459481 10633 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 05:18:09.462117 11073 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:18:09.468044 10943 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d92df9b3deea4f8381e98c50c78938dd [term 1 FOLLOWER]: Refusing update from remote peer 673923c5d7884e938f5d830ff5a83c07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:18:09.468048 10821 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6216c364b55b446eaa89fac9c57f7d3d [term 1 FOLLOWER]: Refusing update from remote peer 673923c5d7884e938f5d830ff5a83c07: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:18:09.468842 11049 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6216c364b55b446eaa89fac9c57f7d3d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:18:09.469097 11069 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d92df9b3deea4f8381e98c50c78938dd" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:18:09.473371 10997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6216c364b55b446eaa89fac9c57f7d3d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 673923c5d7884e938f5d830ff5a83c07. Latest consensus state: current_term: 1 leader_uuid: "673923c5d7884e938f5d830ff5a83c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673923c5d7884e938f5d830ff5a83c07" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37225 } } peers { permanent_uuid: "6216c364b55b446eaa89fac9c57f7d3d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 } } peers { permanent_uuid: "d92df9b3deea4f8381e98c50c78938dd" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } } }
I20260811 05:18:09.473389 11024 sys_catalog.cc:455] T 00000000000000000000000000000000 P d92df9b3deea4f8381e98c50c78938dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 673923c5d7884e938f5d830ff5a83c07. Latest consensus state: current_term: 1 leader_uuid: "673923c5d7884e938f5d830ff5a83c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673923c5d7884e938f5d830ff5a83c07" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37225 } } peers { permanent_uuid: "6216c364b55b446eaa89fac9c57f7d3d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 } } peers { permanent_uuid: "d92df9b3deea4f8381e98c50c78938dd" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } } }
I20260811 05:18:09.473495 10997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6216c364b55b446eaa89fac9c57f7d3d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:18:09.473513 11024 sys_catalog.cc:458] T 00000000000000000000000000000000 P d92df9b3deea4f8381e98c50c78938dd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:18:09.479883 10752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.459891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58094 (local address 127.7.67.190:37225)
0811 05:18:09.460008 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.460013 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.468829 (+ 8816us) server_negotiation.cc:407] Connection header received
0811 05:18:09.469038 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.469042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.469095 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.469150 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.470276 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.471008 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.473614 (+ 2606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.473877 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.473939 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:09.475760 (+ 1821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:09.475764 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:09.475767 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:09.475793 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:09.477674 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.478167 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.478170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.478172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.478278 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.478774 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.478779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.478782 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.479037 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:09.479158 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:09.479593 (+ 435us) server_negotiation.cc:299] Negotiation successful
0811 05:18:09.479715 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 05:18:09.485787 11049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "673923c5d7884e938f5d830ff5a83c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673923c5d7884e938f5d830ff5a83c07" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37225 } } peers { permanent_uuid: "6216c364b55b446eaa89fac9c57f7d3d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 } } peers { permanent_uuid: "d92df9b3deea4f8381e98c50c78938dd" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } } }
I20260811 05:18:09.485949 11049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07 [sys.catalog]: This master's current role is: LEADER
I20260811 05:18:09.487358 10997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6216c364b55b446eaa89fac9c57f7d3d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "673923c5d7884e938f5d830ff5a83c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673923c5d7884e938f5d830ff5a83c07" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37225 } } peers { permanent_uuid: "6216c364b55b446eaa89fac9c57f7d3d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 } } peers { permanent_uuid: "d92df9b3deea4f8381e98c50c78938dd" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } } }
I20260811 05:18:09.487480 10997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6216c364b55b446eaa89fac9c57f7d3d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:18:09.487934 10993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.461362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56880 (local address 127.7.67.188:42917)
0811 05:18:09.461474 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.461479 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.476593 (+ 15114us) server_negotiation.cc:407] Connection header received
0811 05:18:09.476811 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.476817 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.476877 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.476934 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.479958 (+ 3024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.480679 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.481629 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.481974 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.482081 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:09.484047 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:09.484053 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:09.484059 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:09.484091 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:09.485949 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.486484 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.486487 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.486489 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.486585 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.486924 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.486927 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.486929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.487142 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:09.487256 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:09.487627 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 05:18:09.487754 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 05:18:09.488190 11082 mvcc.cc:205] Tried to move back new op lower bound from 7317198804852408320 to 7317198804803600384. Current Snapshot: MvccSnapshot[applied={T|T < 7317198804852408320}]
I20260811 05:18:09.489558 11073 catalog_manager.cc:1347] Generated new cluster ID: f2b7dbae09174eaeb332603ad3a1e27e
I20260811 05:18:09.489663 11073 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:18:09.493031 10879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:09.460639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55202 (local address 127.7.67.189:45783)
0811 05:18:09.460757 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.460762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.472325 (+ 11563us) server_negotiation.cc:407] Connection header received
0811 05:18:09.472502 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.472505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.472576 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.472629 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.474408 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.475210 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.477836 (+ 2626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:09.478189 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:09.478264 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:09.481678 (+ 3414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:09.481682 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:09.481684 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:09.481712 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:09.483577 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.484090 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.484093 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.484096 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.484193 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:09.484575 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:09.484579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:09.484582 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:09.484844 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:09.484971 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:09.492760 (+ 7789us) server_negotiation.cc:299] Negotiation successful
0811 05:18:09.492890 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 05:18:09.497180 11069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "673923c5d7884e938f5d830ff5a83c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673923c5d7884e938f5d830ff5a83c07" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37225 } } peers { permanent_uuid: "6216c364b55b446eaa89fac9c57f7d3d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 } } peers { permanent_uuid: "d92df9b3deea4f8381e98c50c78938dd" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } } }
I20260811 05:18:09.497316 11069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07 [sys.catalog]: This master's current role is: LEADER
W20260811 05:18:09.497913 10913 tablet.cc:2369] T 00000000000000000000000000000000 P d92df9b3deea4f8381e98c50c78938dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 05:18:09.499027 11024 sys_catalog.cc:455] T 00000000000000000000000000000000 P d92df9b3deea4f8381e98c50c78938dd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "673923c5d7884e938f5d830ff5a83c07" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "673923c5d7884e938f5d830ff5a83c07" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 37225 } } peers { permanent_uuid: "6216c364b55b446eaa89fac9c57f7d3d" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 45783 } } peers { permanent_uuid: "d92df9b3deea4f8381e98c50c78938dd" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 42917 } } }
I20260811 05:18:09.499150 11024 sys_catalog.cc:458] T 00000000000000000000000000000000 P d92df9b3deea4f8381e98c50c78938dd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:18:09.500595 11073 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:18:09.501713 11073 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:18:09.511957 11080 mvcc.cc:205] Tried to move back new op lower bound from 7317198804852408320 to 7317198804803600384. Current Snapshot: MvccSnapshot[applied={T|T < 7317198804852408320}]
I20260811 05:18:09.519691 11073 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 673923c5d7884e938f5d830ff5a83c07: Generated new TSK 0
I20260811 05:18:09.520493 11073 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 05:18:09.547962 10752 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:09.546169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58108 (local address 127.7.67.190:37225)
0811 05:18:09.546311 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.546316 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.547010 (+ 694us) server_negotiation.cc:407] Connection header received
0811 05:18:09.547192 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.547196 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.547246 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.547299 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.547822 (+ 523us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58108: BlockingRecv error: recv got EOF from 127.0.0.1:58108 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 05:18:09.547976 10993 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:09.546253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56888 (local address 127.7.67.188:42917)
0811 05:18:09.546366 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.546370 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.547057 (+ 687us) server_negotiation.cc:407] Connection header received
0811 05:18:09.547239 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.547243 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.547296 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.547366 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.547856 (+ 490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56888: BlockingRecv error: recv got EOF from 127.0.0.1:56888 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 05:18:09.547968 10879 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:09.546601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55210 (local address 127.7.67.189:45783)
0811 05:18:09.546722 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.546727 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.546755 (+ 28us) server_negotiation.cc:407] Connection header received
0811 05:18:09.546956 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.546961 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.547029 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.547107 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.547794 (+ 687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55210: BlockingRecv error: recv got EOF from 127.0.0.1:55210 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 05:18:09.551571 10879 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:09.550521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55226 (local address 127.7.67.189:45783)
0811 05:18:09.550669 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.550672 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.550757 (+ 85us) server_negotiation.cc:407] Connection header received
0811 05:18:09.550917 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.550919 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.550978 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.551048 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.551468 (+ 420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55226: BlockingRecv error: recv got EOF from 127.0.0.1:55226 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 05:18:09.555536 10879 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:09.554753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55228 (local address 127.7.67.189:45783)
0811 05:18:09.554874 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.554877 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.554888 (+ 11us) server_negotiation.cc:407] Connection header received
0811 05:18:09.554924 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.554926 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.554962 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.555052 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.555438 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55228: BlockingRecv error: recv got EOF from 127.0.0.1:55228 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 05:18:09.560238 10879 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:09.559232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55238 (local address 127.7.67.189:45783)
0811 05:18:09.559346 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.559350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.559496 (+ 146us) server_negotiation.cc:407] Connection header received
0811 05:18:09.559610 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.559612 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.559650 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.559714 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.560144 (+ 430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55238: BlockingRecv error: recv got EOF from 127.0.0.1:55238 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 05:18:09.567098 10879 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:09.566107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55248 (local address 127.7.67.189:45783)
0811 05:18:09.566262 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.566265 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.566359 (+ 94us) server_negotiation.cc:407] Connection header received
0811 05:18:09.566481 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.566483 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.566520 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.566583 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.566998 (+ 415us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55248: BlockingRecv error: recv got EOF from 127.0.0.1:55248 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260811 05:18:09.578155 10879 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:09.577113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55264 (local address 127.7.67.189:45783)
0811 05:18:09.577232 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.577236 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.577383 (+ 147us) server_negotiation.cc:407] Connection header received
0811 05:18:09.577510 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.577512 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.577553 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.577613 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.578052 (+ 439us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55264: BlockingRecv error: recv got EOF from 127.0.0.1:55264 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 05:18:09.597117 10879 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:09.596039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55268 (local address 127.7.67.189:45783)
0811 05:18:09.596172 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.596177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.596330 (+ 153us) server_negotiation.cc:407] Connection header received
0811 05:18:09.596429 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.596432 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.596488 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.596594 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.596980 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55268: BlockingRecv error: recv got EOF from 127.0.0.1:55268 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 05:18:09.632468 10879 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:09.631251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55282 (local address 127.7.67.189:45783)
0811 05:18:09.631418 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.631424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.631570 (+ 146us) server_negotiation.cc:407] Connection header received
0811 05:18:09.631701 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.631705 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.631776 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.631879 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.632341 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55282: BlockingRecv error: recv got EOF from 127.0.0.1:55282 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 05:18:09.699883 10879 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:09.698750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55288 (local address 127.7.67.189:45783)
0811 05:18:09.698933 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.698937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.699018 (+ 81us) server_negotiation.cc:407] Connection header received
0811 05:18:09.699148 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.699151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.699206 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.699283 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.699745 (+ 462us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55288: BlockingRecv error: recv got EOF from 127.0.0.1:55288 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260811 05:18:09.831372 10879 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:09.830129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55302 (local address 127.7.67.189:45783)
0811 05:18:09.830312 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:09.830316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:09.830475 (+ 159us) server_negotiation.cc:407] Connection header received
0811 05:18:09.830606 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:09.830609 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:09.830667 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:09.830739 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:09.831206 (+ 467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55302: BlockingRecv error: recv got EOF from 127.0.0.1:55302 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 05:18:10.090785 10879 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:10.089594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55310 (local address 127.7.67.189:45783)
0811 05:18:10.089774 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:10.089779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:10.089894 (+ 115us) server_negotiation.cc:407] Connection header received
0811 05:18:10.090025 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:10.090028 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:10.090079 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:10.090154 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:10.090648 (+ 494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55310: BlockingRecv error: recv got EOF from 127.0.0.1:55310 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 05:18:10.299866 11018 catalog_manager.cc:1259] Loaded cluster ID: f2b7dbae09174eaeb332603ad3a1e27e
I20260811 05:18:10.299952 11018 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6216c364b55b446eaa89fac9c57f7d3d: loading cluster ID for follower catalog manager: success
I20260811 05:18:10.301760 11018 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6216c364b55b446eaa89fac9c57f7d3d: acquiring CA information for follower catalog manager: success
I20260811 05:18:10.303012 11018 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6216c364b55b446eaa89fac9c57f7d3d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 05:18:10.323545 11042 catalog_manager.cc:1259] Loaded cluster ID: f2b7dbae09174eaeb332603ad3a1e27e
I20260811 05:18:10.323650 11042 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d92df9b3deea4f8381e98c50c78938dd: loading cluster ID for follower catalog manager: success
I20260811 05:18:10.325397 11042 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d92df9b3deea4f8381e98c50c78938dd: acquiring CA information for follower catalog manager: success
I20260811 05:18:10.326567 11042 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d92df9b3deea4f8381e98c50c78938dd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 05:18:10.609189 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:10.604947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55312 (local address 127.7.67.189:45783)
0811 05:18:10.605201 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:10.605206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:10.605322 (+ 116us) server_negotiation.cc:407] Connection header received
0811 05:18:10.605519 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:10.605522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:10.605657 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:10.605780 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:18:10.606077 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:10.606606 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:10.607487 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:10.607719 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:10.607843 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:10.608208 (+ 365us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:18:10.608762 (+ 554us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:18:10.608900 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:10.608966 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 05:18:10.609024 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":71,"threads_started":1}
I20260811 05:18:10.615705 11202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:10.612384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56898 (local address 127.7.67.188:42917)
0811 05:18:10.612690 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:10.612694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:10.612707 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:18:10.612789 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:10.612791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:10.612916 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:10.613014 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:18:10.613205 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:10.613697 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:10.614466 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:10.614677 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:10.614752 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:10.615006 (+ 254us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:18:10.615386 (+ 380us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:18:10.615475 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:10.615544 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 05:18:10.615591 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":75,"threads_started":1}
I20260811 05:18:10.622874 11190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:10.618778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55316 (local address 127.7.67.189:45783)
0811 05:18:10.618897 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:10.618903 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:10.619507 (+ 604us) server_negotiation.cc:407] Connection header received
0811 05:18:10.619593 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:10.619596 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:10.619638 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:10.619715 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:18:10.620145 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:10.620913 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:10.621785 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:10.622040 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:10.622143 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:10.622382 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:18:10.622459 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:18:10.622526 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:10.622675 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 05:18:10.622724 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 05:18:10.623510 11202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:10.619227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56904 (local address 127.7.67.188:42917)
0811 05:18:10.619378 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:10.619384 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:10.619681 (+ 297us) server_negotiation.cc:407] Connection header received
0811 05:18:10.619772 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:10.619777 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:10.619828 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:10.619908 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:18:10.620240 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:10.620807 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:10.622104 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:10.622443 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:10.622557 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:10.622883 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:18:10.622956 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:18:10.623029 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:10.623316 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 05:18:10.623378 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 05:18:10.623909 11214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:10.618720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58118 (local address 127.7.67.190:37225)
0811 05:18:10.619073 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:10.619077 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:10.619354 (+ 277us) server_negotiation.cc:407] Connection header received
0811 05:18:10.619532 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:10.619536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:10.619700 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:10.619814 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:18:10.620062 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:10.620913 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:10.622086 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:10.622421 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:10.622534 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:10.622870 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:18:10.623304 (+ 434us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:18:10.623404 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:10.623692 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 05:18:10.623757 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":68,"threads_started":1}
I20260811 05:18:10.627175 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 10633
I20260811 05:18:10.644218 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 10763
I20260811 05:18:10.653863 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 10881
2026-08-11T05:18:10Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3472 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 05:18:10 dist-test-slave-04bb krb5kdc[11219](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 05:18:10 dist-test-slave-04bb krb5kdc[11219](info): set up 2 sockets
Aug 11 05:18:10 dist-test-slave-04bb krb5kdc[11219](info): commencing operation
krb5kdc: starting...
W20260811 05:18:11.937533 7438 mini_kdc.cc:121] Time spent starting KDC: real 1.254s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/krb5kdc/test-user.keytab.
Aug 11 05:18:12 dist-test-slave-04bb krb5kdc[11219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T05:18:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T05:18:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.7.67.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
Entry for principal kudu/127.7.67.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab.
I20260811 05:18:12.067092 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:45861
--webserver_interface=127.7.67.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:41865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:45861,127.7.67.189:44157,127.7.67.188:39185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:18:12.188938 11240 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 05:18:12.189226 11240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:18:12.189287 11240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:18:12.189393 11240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:18:12.193933 11240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:18:12.193997 11240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:18:12.194018 11240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:18:12.194062 11240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:18:12.194098 11240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:18:12.199405 11240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:41865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:45861,127.7.67.189:44157,127.7.67.188:39185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.190.keytab
--principal=kudu/127.7.67.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.190:45861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.7.67.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:18:12.200596 11240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:18:12.201697 11240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:18:12.210632 11248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:18:12.210726 11240 server_base.cc:1034] running on GCE node
W20260811 05:18:12.210620 11251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:18:12.210625 11246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:18:12.211463 11240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:18:12.212563 11240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:18:12.213744 11240 hybrid_clock.cc:648] HybridClock initialized: now 1786425492213719 us; error 38 us; skew 500 ppm
Aug 11 05:18:12 dist-test-slave-04bb krb5kdc[11219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425492, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:18:12.216643 11240 init.cc:377] Logged in from keytab as kudu/127.7.67.190@KRBTEST.COM (short username kudu)
I20260811 05:18:12.217608 11240 webserver.cc:460] Webserver started at http://127.7.67.190:43677/ using document root <none> and password file <none>
I20260811 05:18:12.218243 11240 fs_manager.cc:359] Metadata directory not provided
I20260811 05:18:12.218341 11240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:18:12.218587 11240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:18:12.220801 11240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-0/data/instance:
uuid: "f2d4d27b157849a19a0d8302b5bd9cb2"
format_stamp: "Formatted at 2026-08-11 05:18:12 on dist-test-slave-04bb"
server_key: "4592983e5e2413427ecd7757c2270616"
server_key_iv: "8a59a0049e1d1375b231c353bf1151fb"
server_key_version: "encryptionkey@0"
I20260811 05:18:12.221390 11240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-0/wal/instance:
uuid: "f2d4d27b157849a19a0d8302b5bd9cb2"
format_stamp: "Formatted at 2026-08-11 05:18:12 on dist-test-slave-04bb"
server_key: "4592983e5e2413427ecd7757c2270616"
server_key_iv: "8a59a0049e1d1375b231c353bf1151fb"
server_key_version: "encryptionkey@0"
I20260811 05:18:12.225328 11240 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 05:18:12.227809 11258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:18:12.229235 11240 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:18:12.229409 11240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-0/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-0/wal
uuid: "f2d4d27b157849a19a0d8302b5bd9cb2"
format_stamp: "Formatted at 2026-08-11 05:18:12 on dist-test-slave-04bb"
server_key: "4592983e5e2413427ecd7757c2270616"
server_key_iv: "8a59a0049e1d1375b231c353bf1151fb"
server_key_version: "encryptionkey@0"
I20260811 05:18:12.229529 11240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:18:12.253110 11240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:18:12.256102 11240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:18:12.256315 11240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:18:12.263593 11240 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.190:45861
I20260811 05:18:12.263891 11337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.190:45861 every 8 connection(s)
I20260811 05:18:12.264720 11240 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-0/data/info.pb
I20260811 05:18:12.267602 11338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:18:12.271533 11338 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45861 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:18:12.274477 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 11240
I20260811 05:18:12.274616 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-0/wal/instance
I20260811 05:18:12.274955 7438 external_mini_cluster.cc:1428] Setting key 6fb8b214740e396854e75d7de80d2c3c
Aug 11 05:18:12 dist-test-slave-04bb krb5kdc[11219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425492, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.7.67.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.189@KRBTEST.COM" created.
I20260811 05:18:12.287359 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.273765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45861 (local address 127.0.0.1:47920)
0811 05:18:12.274247 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.274291 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 05:18:12.275339 (+ 1048us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:18:12.276055 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:18:12.276060 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:18:12.276584 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:18:12.276850 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:12.276871 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.278018 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.278021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:12.278753 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:12.278763 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.279478 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.279482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:12.279592 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:12.280238 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:18:12.280255 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:18:12.282190 (+ 1935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:18:12.284368 (+ 2178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:12.284381 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:12.284394 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:12.284751 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:12.285068 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:12.285070 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:12.285072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:12.285187 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:12.285888 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:18:12.285893 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:18:12.286031 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 05:18:12.286170 (+ 139us) client_negotiation.cc:240] Negotiation successful
0811 05:18:12.286587 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":133,"thread_start_us":117,"threads_started":1}
I20260811 05:18:12.287357 11341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.274654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47920 (local address 127.7.67.190:45861)
0811 05:18:12.274962 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:12.274974 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:12.275021 (+ 47us) server_negotiation.cc:407] Connection header received
0811 05:18:12.275661 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:12.275666 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:12.275857 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:12.276054 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:12.277019 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.277889 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.278918 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.279297 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.279493 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:12.282350 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:12.282366 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:12.282376 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:12.282423 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:12.284235 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.284858 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.284864 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.284865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.284978 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.285278 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.285280 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.285281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.285770 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:12.285883 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:12.286275 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 05:18:12.286496 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":191,"thread_start_us":95,"threads_started":1}
I20260811 05:18:12.289784 11338 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:18:12.290601 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.290339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:44157 (local address 127.0.0.1:50076)
0811 05:18:12.290471 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.290541 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.189:44157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 05:18:12.290832 11266 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.67.189:44157: connect: Connection refused (error 111)
I20260811 05:18:12.291673 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.291457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:44157 (local address 127.0.0.1:50078)
0811 05:18:12.291571 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.291622 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.189:44157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 05:18:12.291849 11338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.189:44157: Network error: Client connection negotiation failed: client connection to 127.7.67.189:44157: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.189.keytab.
Entry for principal kudu/127.7.67.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.189.keytab.
I20260811 05:18:12.304103 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.189:44157
--webserver_interface=127.7.67.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.189.keytab
--principal=kudu/127.7.67.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:41865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:45861,127.7.67.189:44157,127.7.67.188:39185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 05:18:12.341076 11338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 } attempt: 1
I20260811 05:18:12.342064 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.341730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:44157 (local address 127.0.0.1:50088)
0811 05:18:12.341911 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.341979 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.189:44157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 05:18:12.342865 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.342645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:44157 (local address 127.0.0.1:50094)
0811 05:18:12.342750 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.342796 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.189:44157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 05:18:12.343039 11338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.189:44157: Network error: Client connection negotiation failed: client connection to 127.7.67.189:44157: connect: Connection refused (error 111)
I20260811 05:18:12.411214 11338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 } attempt: 2
I20260811 05:18:12.412143 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.411828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:44157 (local address 127.0.0.1:50096)
0811 05:18:12.411996 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.412063 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.189:44157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 05:18:12.412954 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.412740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:44157 (local address 127.0.0.1:50102)
0811 05:18:12.412850 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.412896 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.189:44157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 05:18:12.413112 11338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.189:44157: Network error: Client connection negotiation failed: client connection to 127.7.67.189:44157: connect: Connection refused (error 111)
W20260811 05:18:12.425339 11347 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 05:18:12.425669 11347 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:18:12.425721 11347 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:18:12.425796 11347 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 05:18:12.430052 11347 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:18:12.430112 11347 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:18:12.430155 11347 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:18:12.430191 11347 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 05:18:12.430227 11347 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:18:12.435879 11347 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:41865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:45861,127.7.67.189:44157,127.7.67.188:39185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.189.keytab
--principal=kudu/127.7.67.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.189:44157
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.7.67.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11347
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:18:12.437119 11347 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:18:12.438402 11347 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:18:12.447899 11355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:18:12.447909 11363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:18:12.447919 11358 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:18:12.448215 11347 server_base.cc:1034] running on GCE node
I20260811 05:18:12.448930 11347 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:18:12.450079 11347 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:18:12.451274 11347 hybrid_clock.cc:648] HybridClock initialized: now 1786425492451240 us; error 48 us; skew 500 ppm
Aug 11 05:18:12 dist-test-slave-04bb krb5kdc[11219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425492, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:18:12.454155 11347 init.cc:377] Logged in from keytab as kudu/127.7.67.189@KRBTEST.COM (short username kudu)
I20260811 05:18:12.455063 11347 webserver.cc:460] Webserver started at http://127.7.67.189:40623/ using document root <none> and password file <none>
I20260811 05:18:12.455657 11347 fs_manager.cc:359] Metadata directory not provided
I20260811 05:18:12.455753 11347 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:18:12.456007 11347 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:18:12.458158 11347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-1/data/instance:
uuid: "4d6973e849ab43478a43d575c20ec7ab"
format_stamp: "Formatted at 2026-08-11 05:18:12 on dist-test-slave-04bb"
server_key: "6b5ca25defbb11700b598b9dbce71708"
server_key_iv: "8da5c67f7f732ca3bd07eac451994d48"
server_key_version: "encryptionkey@0"
I20260811 05:18:12.458755 11347 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-1/wal/instance:
uuid: "4d6973e849ab43478a43d575c20ec7ab"
format_stamp: "Formatted at 2026-08-11 05:18:12 on dist-test-slave-04bb"
server_key: "6b5ca25defbb11700b598b9dbce71708"
server_key_iv: "8da5c67f7f732ca3bd07eac451994d48"
server_key_version: "encryptionkey@0"
I20260811 05:18:12.462632 11347 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 05:18:12.465137 11371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:18:12.466455 11347 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 05:18:12.466604 11347 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-1/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-1/wal
uuid: "4d6973e849ab43478a43d575c20ec7ab"
format_stamp: "Formatted at 2026-08-11 05:18:12 on dist-test-slave-04bb"
server_key: "6b5ca25defbb11700b598b9dbce71708"
server_key_iv: "8da5c67f7f732ca3bd07eac451994d48"
server_key_version: "encryptionkey@0"
I20260811 05:18:12.466725 11347 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:18:12.486289 11347 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:18:12.489614 11347 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:18:12.489836 11347 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:18:12.497332 11347 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.189:44157
I20260811 05:18:12.497637 11448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.189:44157 every 8 connection(s)
I20260811 05:18:12.498383 11347 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-1/data/info.pb
I20260811 05:18:12.501489 11449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:18:12.502010 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 11347
I20260811 05:18:12.502154 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-1/wal/instance
I20260811 05:18:12.502497 7438 external_mini_cluster.cc:1428] Setting key 41768877c5913b5a2173a1b796cd3d22
I20260811 05:18:12.504293 11338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 } attempt: 3
I20260811 05:18:12.507416 11449 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45861 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.7.67.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.7.67.188@KRBTEST.COM" created.
Aug 11 05:18:12 dist-test-slave-04bb krb5kdc[11219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425492, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for kudu/127.7.67.189@KRBTEST.COM
Aug 11 05:18:12 dist-test-slave-04bb krb5kdc[11219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425492, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.189@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:18:12.521767 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.504912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:44157 (local address 127.0.0.1:50118)
0811 05:18:12.505151 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.505171 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 05:18:12.505273 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:18:12.508146 (+ 2873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:18:12.508150 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:18:12.508378 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:18:12.508558 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:12.508568 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.510658 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.510662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:12.511385 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:12.511397 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.512133 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.512137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:12.512237 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:12.512860 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:18:12.512873 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:18:12.516990 (+ 4117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:18:12.519490 (+ 2500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:12.519494 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:12.519496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:12.519928 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:12.520341 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:12.520346 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:12.520348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:12.520445 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:12.521219 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:18:12.521222 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:18:12.521325 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 05:18:12.521472 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 05:18:12.521606 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260811 05:18:12.522627 11341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.509444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47922 (local address 127.7.67.190:45861)
0811 05:18:12.509584 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:12.509588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:12.509949 (+ 361us) server_negotiation.cc:407] Connection header received
0811 05:18:12.510186 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:12.510189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:12.510236 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:12.510330 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:12.511173 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.512014 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.513526 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.514365 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.514457 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:12.518045 (+ 3588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:12.518052 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:12.518058 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:12.518384 (+ 326us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:12.520272 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.520894 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.520898 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.520899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.520980 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.521376 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.521380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.521382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.521650 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:12.521768 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:12.522381 (+ 613us) server_negotiation.cc:299] Negotiation successful
0811 05:18:12.522491 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 05:18:12.523389 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.509424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45861 (local address 127.0.0.1:47922)
0811 05:18:12.509812 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.509873 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 05:18:12.510035 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:18:12.510387 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:18:12.510393 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:18:12.510861 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:18:12.511018 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:12.511026 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.512189 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.512195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:12.513310 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:12.513327 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.514516 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.514519 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:12.514601 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:12.515078 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:18:12.515094 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:18:12.517875 (+ 2781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:18:12.520397 (+ 2522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:12.520408 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:12.520418 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:12.520767 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:12.521102 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:12.521108 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:12.521113 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:12.521228 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:12.521784 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:18:12.521792 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:18:12.521953 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 05:18:12.522098 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 05:18:12.522426 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":86,"thread_start_us":116,"threads_started":1}
I20260811 05:18:12.523900 11453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.505565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50118 (local address 127.7.67.189:44157)
0811 05:18:12.506036 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:12.506059 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:12.506111 (+ 52us) server_negotiation.cc:407] Connection header received
0811 05:18:12.507224 (+ 1113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:12.507254 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:12.507745 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:12.508201 (+ 456us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:12.508751 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.510508 (+ 1757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.511518 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.512005 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.512181 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:12.517141 (+ 4960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:12.517148 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:12.517162 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:12.517195 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:12.519327 (+ 2132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.520057 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.520067 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.520077 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.520226 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.520582 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.520585 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.520587 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.521114 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:12.521254 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:12.522052 (+ 798us) server_negotiation.cc:299] Negotiation successful
0811 05:18:12.523733 (+ 1681us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":118,"threads_started":1}
I20260811 05:18:12.525460 11449 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:18:12.526130 11338 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:18:12.526882 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.526598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:39185 (local address 127.0.0.1:43448)
0811 05:18:12.526726 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.526784 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 05:18:12.527670 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.527382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:39185 (local address 127.0.0.1:43460)
0811 05:18:12.527525 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.527575 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260811 05:18:12.527804 11338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.188:39185: Network error: Client connection negotiation failed: client connection to 127.7.67.188:39185: connect: Connection refused (error 111)
Aug 11 05:18:12 dist-test-slave-04bb krb5kdc[11219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425492, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.189@KRBTEST.COM for kudu/127.7.67.189@KRBTEST.COM
I20260811 05:18:12.533818 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.526006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:44157 (local address 127.0.0.1:50134)
0811 05:18:12.526151 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.526166 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:18:12.526245 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:18:12.526483 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:18:12.526485 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:18:12.526681 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:18:12.526772 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:12.526778 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.527650 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.527654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:12.528384 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:12.528394 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.529026 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.529029 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:12.529091 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:12.529610 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:18:12.529624 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:18:12.530841 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:18:12.532372 (+ 1531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:12.532375 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:12.532376 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:12.532699 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:12.532966 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:12.532969 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:12.532970 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:12.533046 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:12.533409 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:18:12.533411 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:18:12.533486 (+ 75us) client_negotiation.cc:769] Sending connection context
0811 05:18:12.533598 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 05:18:12.533711 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260811 05:18:12.534003 11453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.526038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50134 (local address 127.7.67.189:44157)
0811 05:18:12.526136 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:12.526140 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:12.526207 (+ 67us) server_negotiation.cc:407] Connection header received
0811 05:18:12.526348 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:12.526351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:12.526387 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:12.526473 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:12.526902 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.527542 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.528491 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.528919 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.529009 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:12.530965 (+ 1956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:12.530968 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:12.530970 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:12.530991 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:12.532248 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.532796 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.532799 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.532801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.532890 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.533113 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.533115 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.533116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.533275 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:12.533401 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:12.533672 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 05:18:12.533769 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.7.67.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.188.keytab.
Entry for principal kudu/127.7.67.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.188.keytab.
I20260811 05:18:12.534741 11449 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } has no permanent_uuid. Determining permanent_uuid...
I20260811 05:18:12.535414 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.535170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:39185 (local address 127.0.0.1:43466)
0811 05:18:12.535282 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.535364 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 05:18:12.535575 7438 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
/tmp/dist-test-tasktf25Hl/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.188:39185
--webserver_interface=127.7.67.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.188.keytab
--principal=kudu/127.7.67.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.7.67.148:41865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.7.67.190:45861,127.7.67.189:44157,127.7.67.188:39185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 05:18:12.536041 11373 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.7.67.188:39185: connect: Connection refused (error 111)
I20260811 05:18:12.536979 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.536771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:39185 (local address 127.0.0.1:43482)
0811 05:18:12.536877 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.536932 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 05:18:12.537147 11449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.188:39185: Network error: Client connection negotiation failed: client connection to 127.7.67.188:39185: connect: Connection refused (error 111)
I20260811 05:18:12.559103 11338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } attempt: 1
I20260811 05:18:12.559993 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.559685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:39185 (local address 127.0.0.1:43488)
0811 05:18:12.559834 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.559915 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 05:18:12.560820 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.560604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:39185 (local address 127.0.0.1:43504)
0811 05:18:12.560712 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.560764 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 05:18:12.560987 11338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.188:39185: Network error: Client connection negotiation failed: client connection to 127.7.67.188:39185: connect: Connection refused (error 111)
I20260811 05:18:12.586408 11449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } attempt: 1
I20260811 05:18:12.587354 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.587038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:39185 (local address 127.0.0.1:43514)
0811 05:18:12.587201 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.587267 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 05:18:12.588128 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.587921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:39185 (local address 127.0.0.1:43520)
0811 05:18:12.588026 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.588070 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 05:18:12.588284 11449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.188:39185: Network error: Client connection negotiation failed: client connection to 127.7.67.188:39185: connect: Connection refused (error 111)
I20260811 05:18:12.636143 11338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } attempt: 2
I20260811 05:18:12.637099 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.636777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:39185 (local address 127.0.0.1:43528)
0811 05:18:12.636948 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.637016 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 05:18:12.637866 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.637638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:39185 (local address 127.0.0.1:43544)
0811 05:18:12.637750 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.637804 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 05:18:12.637995 11338 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.188:39185: Network error: Client connection negotiation failed: client connection to 127.7.67.188:39185: connect: Connection refused (error 111)
W20260811 05:18:12.652623 11461 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 05:18:12.652931 11461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 05:18:12.652992 11461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 05:18:12.653069 11461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 05:18:12.656486 11449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } attempt: 2
W20260811 05:18:12.657272 11461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 05:18:12.657359 11461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 05:18:12.657404 11461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 05:18:12.657439 11461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
I20260811 05:18:12.657425 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.657083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:39185 (local address 127.0.0.1:43558)
0811 05:18:12.657238 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.657304 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 05:18:12.657476 11461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 05:18:12.658104 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.657916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:39185 (local address 127.0.0.1:43568)
0811 05:18:12.658014 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.658057 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.7.67.188:39185: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 05:18:12.658236 11449 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.7.67.188:39185: Network error: Client connection negotiation failed: client connection to 127.7.67.188:39185: connect: Connection refused (error 111)
I20260811 05:18:12.663172 11461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.7.67.148:41865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.7.67.190:45861,127.7.67.189:44157,127.7.67.188:39185
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/krb5kdc/kudu_127.7.67.188.keytab
--principal=kudu/127.7.67.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.7.67.188:39185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.7.67.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 04:27:57 UTC on ccycloud.quasar-xvkbbt.root.comops.site
build id 4579723
I20260811 05:18:12.664289 11461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 05:18:12.665416 11461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 05:18:12.674116 11469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 05:18:12.674124 11467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:18:12.674376 11461 server_base.cc:1034] running on GCE node
W20260811 05:18:12.674506 11472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 05:18:12.675045 11461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 05:18:12.676141 11461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 05:18:12.677352 11461 hybrid_clock.cc:648] HybridClock initialized: now 1786425492677326 us; error 30 us; skew 500 ppm
Aug 11 05:18:12 dist-test-slave-04bb krb5kdc[11219](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425492, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 05:18:12.680212 11461 init.cc:377] Logged in from keytab as kudu/127.7.67.188@KRBTEST.COM (short username kudu)
I20260811 05:18:12.681116 11461 webserver.cc:460] Webserver started at http://127.7.67.188:45343/ using document root <none> and password file <none>
I20260811 05:18:12.681649 11461 fs_manager.cc:359] Metadata directory not provided
I20260811 05:18:12.681710 11461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 05:18:12.681880 11461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 05:18:12.683909 11461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-2/data/instance:
uuid: "128e950eab08434699ff390c66ff01e2"
format_stamp: "Formatted at 2026-08-11 05:18:12 on dist-test-slave-04bb"
server_key: "4de17f1fb6ec9a11e8648d34e22f3e32"
server_key_iv: "f73d8817ff9072cf2f19c26e76b0abea"
server_key_version: "encryptionkey@0"
I20260811 05:18:12.684597 11461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-2/wal/instance:
uuid: "128e950eab08434699ff390c66ff01e2"
format_stamp: "Formatted at 2026-08-11 05:18:12 on dist-test-slave-04bb"
server_key: "4de17f1fb6ec9a11e8648d34e22f3e32"
server_key_iv: "f73d8817ff9072cf2f19c26e76b0abea"
server_key_version: "encryptionkey@0"
I20260811 05:18:12.688221 11461 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 05:18:12.690677 11483 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 05:18:12.691921 11461 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 05:18:12.692057 11461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-2/data,/tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-2/wal
uuid: "128e950eab08434699ff390c66ff01e2"
format_stamp: "Formatted at 2026-08-11 05:18:12 on dist-test-slave-04bb"
server_key: "4de17f1fb6ec9a11e8648d34e22f3e32"
server_key_iv: "f73d8817ff9072cf2f19c26e76b0abea"
server_key_version: "encryptionkey@0"
I20260811 05:18:12.692186 11461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 05:18:12.707026 11461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 05:18:12.710459 11461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 05:18:12.710668 11461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 05:18:12.717451 11461 rpc_server.cc:307] RPC server started. Bound to: 127.7.67.188:39185
I20260811 05:18:12.717777 11570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.7.67.188:39185 every 8 connection(s)
I20260811 05:18:12.718467 11461 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-2/data/info.pb
I20260811 05:18:12.721408 11571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 05:18:12.722841 7438 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu as pid 11461
I20260811 05:18:12.723001 7438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktf25Hl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786425190664461-7438-0/minicluster-data/master-2/wal/instance
I20260811 05:18:12.723338 7438 external_mini_cluster.cc:1428] Setting key 67cb55359cc6b03bc24ea71ec8051418
I20260811 05:18:12.725999 11571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45861 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:18:12 dist-test-slave-04bb krb5kdc[11219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.188@KRBTEST.COM
Aug 11 05:18:12 dist-test-slave-04bb krb5kdc[11219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425492, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.188@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:18:12.737157 11338 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } attempt: 3
I20260811 05:18:12.738217 11341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.727766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47930 (local address 127.7.67.190:45861)
0811 05:18:12.727933 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:12.727938 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:12.728329 (+ 391us) server_negotiation.cc:407] Connection header received
0811 05:18:12.728461 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:12.728464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:12.728514 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:12.728616 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:12.729316 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.730065 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.731269 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.731592 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.731710 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:12.734312 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:12.734316 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:12.734318 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:12.734360 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:12.735688 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.736176 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.736180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.736182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.736267 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.736627 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.736631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.736632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.736835 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:12.736948 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:12.737788 (+ 840us) server_negotiation.cc:299] Negotiation successful
0811 05:18:12.738063 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 05:18:12.738955 11578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.727698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.190:45861 (local address 127.0.0.1:47930)
0811 05:18:12.728181 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.728231 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 05:18:12.728356 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:18:12.728675 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:18:12.728681 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:18:12.729008 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:18:12.729130 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:12.729136 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.730204 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.730207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:12.731129 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:12.731140 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.731753 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.731756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:12.731842 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:12.732451 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:18:12.732469 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:18:12.734123 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:18:12.735803 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:12.735812 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:12.735822 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:12.736063 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:12.736363 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:12.736369 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:12.736373 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:12.736496 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:12.736948 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:18:12.736955 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:18:12.737180 (+ 225us) client_negotiation.cc:769] Sending connection context
0811 05:18:12.737404 (+ 224us) client_negotiation.cc:240] Negotiation successful
0811 05:18:12.737992 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":114,"threads_started":1}
I20260811 05:18:12.738958 11577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.724841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43572 (local address 127.7.67.188:39185)
0811 05:18:12.725272 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:12.725288 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:12.725325 (+ 37us) server_negotiation.cc:407] Connection header received
0811 05:18:12.725963 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:12.725980 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:12.726308 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:12.726693 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:12.727576 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.728923 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.730165 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.730882 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.731061 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:12.733771 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:12.733779 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:12.733792 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:12.733827 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:12.735793 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.736290 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.736297 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.736299 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.736422 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.736780 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.736784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.736786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.737200 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:12.738103 (+ 903us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:12.738208 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 05:18:12.738347 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":128,"threads_started":1}
I20260811 05:18:12.741083 11571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:18:12 dist-test-slave-04bb krb5kdc[11219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425492, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.190@KRBTEST.COM for kudu/127.7.67.188@KRBTEST.COM
Aug 11 05:18:12 dist-test-slave-04bb krb5kdc[11219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425492, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.188@KRBTEST.COM for kudu/127.7.67.189@KRBTEST.COM
Aug 11 05:18:12 dist-test-slave-04bb krb5kdc[11219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.190@KRBTEST.COM
I20260811 05:18:12.748970 11340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.737610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:39185 (local address 127.0.0.1:43580)
0811 05:18:12.737714 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.737729 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:18:12.737798 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:18:12.739562 (+ 1764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:18:12.739565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:18:12.739701 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:18:12.739778 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:12.739783 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.740753 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.740756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:12.741446 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:12.741462 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.742165 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.742169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:12.742247 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:12.742784 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:18:12.742802 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:18:12.745139 (+ 2337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:18:12.747276 (+ 2137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:12.747278 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:12.747280 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:12.747466 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:12.747792 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:12.747795 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:12.747796 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:12.747890 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:12.748361 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:18:12.748364 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:18:12.748458 (+ 94us) client_negotiation.cc:769] Sending connection context
0811 05:18:12.748641 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 05:18:12.748783 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 05:18:12.749009 11577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.737711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43580 (local address 127.7.67.188:39185)
0811 05:18:12.739331 (+ 1620us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:12.739336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:12.739351 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:18:12.739391 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:12.739394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:12.739438 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:12.739527 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:12.739911 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.740647 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.741695 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.742030 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.742464 (+ 434us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:12.745279 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:12.745286 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:12.745292 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:12.745326 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:12.747157 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.747568 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.747572 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.747575 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.747689 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.747997 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.748001 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.748003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.748230 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:12.748354 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:12.748700 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 05:18:12.748822 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1552,"thread_start_us":184,"threads_started":1}
I20260811 05:18:12.749390 11449 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } attempt: 3
I20260811 05:18:12.753440 11338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2: Bootstrap starting.
I20260811 05:18:12.753628 11341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.741803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47938 (local address 127.7.67.190:45861)
0811 05:18:12.742049 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:12.742054 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:12.742069 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:18:12.742170 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:12.742174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:12.742215 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:12.742319 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:12.743090 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.743693 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.744958 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.745254 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.745383 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:12.749139 (+ 3756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:12.749143 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:12.749145 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:12.749172 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:12.751277 (+ 2105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.751775 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.751778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.751780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.751897 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.752212 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.752218 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.752223 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.752431 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:12.752651 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:12.753378 (+ 727us) server_negotiation.cc:299] Negotiation successful
0811 05:18:12.753488 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 05:18:12.753922 11578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.741597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.189:44157 (local address 127.0.0.1:50136)
0811 05:18:12.741960 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.741974 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:18:12.742054 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:18:12.742460 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:18:12.742463 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:18:12.742637 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:18:12.742761 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:12.742769 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.743827 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.743833 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:12.744860 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:12.744875 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.745619 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.745626 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:12.745732 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:12.746365 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:18:12.746385 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:18:12.748644 (+ 2259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:18:12.751435 (+ 2791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:12.751438 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:12.751440 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:12.751698 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:12.752028 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:12.752031 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:12.752033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:12.752150 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:12.753387 (+ 1237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:18:12.753390 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:18:12.753485 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 05:18:12.753598 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 05:18:12.753727 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299}
I20260811 05:18:12.754511 11453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.741842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50136 (local address 127.7.67.189:44157)
0811 05:18:12.741978 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:12.741984 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:12.742139 (+ 155us) server_negotiation.cc:407] Connection header received
0811 05:18:12.742187 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:12.742190 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:12.742239 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:12.742310 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:12.742912 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.743602 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.745004 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.745388 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.745486 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:12.748815 (+ 3329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:12.748819 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:12.748821 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:12.748862 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:12.751314 (+ 2452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.751824 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.751828 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.751830 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.751922 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.752260 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.752263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.752265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.753247 (+ 982us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:12.753340 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:12.754193 (+ 853us) server_negotiation.cc:299] Negotiation successful
0811 05:18:12.754324 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":28,"server-negotiator.queue_time_us":45}
I20260811 05:18:12.755256 11571 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 05:18:12 dist-test-slave-04bb krb5kdc[11219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425492, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.189@KRBTEST.COM for kudu/127.7.67.188@KRBTEST.COM
I20260811 05:18:12.756492 11338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2: Neither blocks nor log segments found. Creating new log.
I20260811 05:18:12.757521 11338 log.cc:824] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2: Log is configured to *not* fsync() on all Append() calls
I20260811 05:18:12.760469 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.749860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:39185 (local address 127.0.0.1:43586)
0811 05:18:12.749978 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.749993 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 05:18:12.750074 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:18:12.750750 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:18:12.750757 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:18:12.750930 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:18:12.751053 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:12.751062 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.752260 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.752263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:12.752973 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:12.752990 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.753640 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.753645 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:12.753741 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:12.754241 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:18:12.754256 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:18:12.756273 (+ 2017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:18:12.758597 (+ 2324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:12.758601 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:12.758604 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:12.758874 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:12.759196 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:12.759201 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:12.759204 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:12.759309 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:12.759874 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:18:12.759877 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:18:12.759977 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 05:18:12.760091 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 05:18:12.760288 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":15}
I20260811 05:18:12.761161 11581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.750183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43586 (local address 127.7.67.188:39185)
0811 05:18:12.750442 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:12.750456 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:12.750473 (+ 17us) server_negotiation.cc:407] Connection header received
0811 05:18:12.750514 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:12.750520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:12.750566 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:12.751190 (+ 624us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:12.751304 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.752062 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.753113 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.753482 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.755009 (+ 1527us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:12.756637 (+ 1628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:12.756641 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:12.756645 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:12.756673 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:12.758460 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.759011 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.759014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.759016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.759111 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.759441 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.759445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.759447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.759710 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:12.759821 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:12.760841 (+ 1020us) server_negotiation.cc:299] Negotiation successful
0811 05:18:12.760970 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":174}
I20260811 05:18:12.761256 11338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2: No bootstrap required, opened a new log
Aug 11 05:18:12 dist-test-slave-04bb krb5kdc[11219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425492, etypes {rep=17 tkt=17 ses=17}, kudu/127.7.67.188@KRBTEST.COM for kudu/127.7.67.188@KRBTEST.COM
Aug 11 05:18:12 dist-test-slave-04bb krb5kdc[11219](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786425492, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.7.67.189@KRBTEST.COM
I20260811 05:18:12.764202 11449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d6973e849ab43478a43d575c20ec7ab: Bootstrap starting.
I20260811 05:18:12.765498 11338 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45861 } } peers { permanent_uuid: "4d6973e849ab43478a43d575c20ec7ab" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 } } peers { permanent_uuid: "128e950eab08434699ff390c66ff01e2" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } }
I20260811 05:18:12.765759 11338 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2d4d27b157849a19a0d8302b5bd9cb2, State: Initialized, Role: FOLLOWER
I20260811 05:18:12.766160 11578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.755760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.7.67.188:39185 (local address 127.0.0.1:43598)
0811 05:18:12.756428 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0811 05:18:12.756442 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 05:18:12.756559 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 05:18:12.756900 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 05:18:12.756906 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 05:18:12.757068 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0811 05:18:12.757200 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:12.757210 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.758244 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.758247 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:12.759020 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 05:18:12.759029 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.759588 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.759592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 05:18:12.759667 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:12.760140 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 05:18:12.760154 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 05:18:12.762167 (+ 2013us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 05:18:12.764347 (+ 2180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:12.764350 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:12.764352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:12.764670 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:12.764989 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 05:18:12.764992 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 05:18:12.764994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 05:18:12.765073 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 05:18:12.765510 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 05:18:12.765512 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 05:18:12.765616 (+ 104us) client_negotiation.cc:769] Sending connection context
0811 05:18:12.765712 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 05:18:12.765993 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"mutex_wait_us":1}
I20260811 05:18:12.766108 11338 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45861 } } peers { permanent_uuid: "4d6973e849ab43478a43d575c20ec7ab" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 } } peers { permanent_uuid: "128e950eab08434699ff390c66ff01e2" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } }
I20260811 05:18:12.766992 11577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.756022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43598 (local address 127.7.67.188:39185)
0811 05:18:12.756384 (+ 362us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:12.756389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:12.756505 (+ 116us) server_negotiation.cc:407] Connection header received
0811 05:18:12.756673 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:12.756676 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:12.756718 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:12.756791 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:12.757342 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.758093 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.759150 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.759463 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.759864 (+ 401us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:12.762303 (+ 2439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:12.762306 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:12.762309 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:12.762330 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:12.764172 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.764784 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.764786 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.764788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.764886 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.765167 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.765172 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.765174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.765375 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:12.765472 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:12.766707 (+ 1235us) server_negotiation.cc:299] Negotiation successful
0811 05:18:12.766822 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":279,"server-negotiator.queue_time_us":285}
I20260811 05:18:12.767055 11449 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4d6973e849ab43478a43d575c20ec7ab: Neither blocks nor log segments found. Creating new log.
I20260811 05:18:12.767357 11453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.755933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50144 (local address 127.7.67.189:44157)
0811 05:18:12.756035 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:12.756040 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:12.756055 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:18:12.756091 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:12.756094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:12.756137 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:12.756232 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:12.757314 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.758491 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.759270 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.759627 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.759693 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:12.763596 (+ 3903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:12.763599 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:12.763602 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:12.763630 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:12.765391 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.765881 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.765884 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.765886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.765974 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.766306 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.766311 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.766315 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.766533 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:12.766688 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:12.767072 (+ 384us) server_negotiation.cc:299] Negotiation successful
0811 05:18:12.767192 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 05:18:12.767597 11338 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:18:12.767939 11449 log.cc:824] T 00000000000000000000000000000000 P 4d6973e849ab43478a43d575c20ec7ab: Log is configured to *not* fsync() on all Append() calls
I20260811 05:18:12.770320 11571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 128e950eab08434699ff390c66ff01e2: Bootstrap starting.
I20260811 05:18:12.770812 11586 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45861 } } peers { permanent_uuid: "4d6973e849ab43478a43d575c20ec7ab" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 } } peers { permanent_uuid: "128e950eab08434699ff390c66ff01e2" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } } }
I20260811 05:18:12.770941 11586 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:18:12.771869 7438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 05:18:12.772010 11449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4d6973e849ab43478a43d575c20ec7ab: No bootstrap required, opened a new log
I20260811 05:18:12.772832 11571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 128e950eab08434699ff390c66ff01e2: Neither blocks nor log segments found. Creating new log.
I20260811 05:18:12.773106 11303 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47938
I20260811 05:18:12.773360 11303 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 05:18:12.773479 11303 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:18:12.773664 11571 log.cc:824] T 00000000000000000000000000000000 P 128e950eab08434699ff390c66ff01e2: Log is configured to *not* fsync() on all Append() calls
I20260811 05:18:12.775473 11303 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45861 } } peers { permanent_uuid: "4d6973e849ab43478a43d575c20ec7ab" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 } } peers { permanent_uuid: "128e950eab08434699ff390c66ff01e2" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } }
I20260811 05:18:12.775672 11449 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4d6973e849ab43478a43d575c20ec7ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45861 } } peers { permanent_uuid: "4d6973e849ab43478a43d575c20ec7ab" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 } } peers { permanent_uuid: "128e950eab08434699ff390c66ff01e2" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } }
I20260811 05:18:12.775899 11449 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4d6973e849ab43478a43d575c20ec7ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d6973e849ab43478a43d575c20ec7ab, State: Initialized, Role: FOLLOWER
I20260811 05:18:12.776206 11449 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4d6973e849ab43478a43d575c20ec7ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45861 } } peers { permanent_uuid: "4d6973e849ab43478a43d575c20ec7ab" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 } } peers { permanent_uuid: "128e950eab08434699ff390c66ff01e2" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } }
I20260811 05:18:12.773391 11240 master_runner.cc:186] Error getting master registration for 127.7.67.189:44157: OK, instance_id { permanent_uuid: "4d6973e849ab43478a43d575c20ec7ab" instance_seqno: 1786425492495255 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 05:18:12.777163 11240 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 05:18:12.777087 11303 leader_election.cc:288] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2 [CANDIDATE]: Term 1 election: Requested vote from peers 4d6973e849ab43478a43d575c20ec7ab (127.7.67.189:44157), 128e950eab08434699ff390c66ff01e2 (127.7.67.188:39185)
I20260811 05:18:12.777662 11449 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4d6973e849ab43478a43d575c20ec7ab [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:18:12.777760 11405 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4d6973e849ab43478a43d575c20ec7ab"
I20260811 05:18:12.777987 11405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4d6973e849ab43478a43d575c20ec7ab [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:18:12.779392 11405 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4d6973e849ab43478a43d575c20ec7ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2d4d27b157849a19a0d8302b5bd9cb2 in term 1.
I20260811 05:18:12.779963 11266 leader_election.cc:302] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d6973e849ab43478a43d575c20ec7ab, f2d4d27b157849a19a0d8302b5bd9cb2; no voters:
I20260811 05:18:12.780666 11586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2 [term 1 FOLLOWER]: Leader election won for term 1
W20260811 05:18:12.780720 11609 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:18:12.780797 11609 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:18:12.780910 11586 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2 [term 1 LEADER]: Becoming Leader. State: Replica: f2d4d27b157849a19a0d8302b5bd9cb2, State: Running, Role: LEADER
I20260811 05:18:12.781287 11586 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45861 } } peers { permanent_uuid: "4d6973e849ab43478a43d575c20ec7ab" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 } } peers { permanent_uuid: "128e950eab08434699ff390c66ff01e2" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } }
I20260811 05:18:12.782985 11571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 128e950eab08434699ff390c66ff01e2: No bootstrap required, opened a new log
I20260811 05:18:12.787454 11571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 128e950eab08434699ff390c66ff01e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45861 } } peers { permanent_uuid: "4d6973e849ab43478a43d575c20ec7ab" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 } } peers { permanent_uuid: "128e950eab08434699ff390c66ff01e2" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } }
I20260811 05:18:12.787724 11571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 128e950eab08434699ff390c66ff01e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 128e950eab08434699ff390c66ff01e2, State: Initialized, Role: FOLLOWER
I20260811 05:18:12.788086 11571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 128e950eab08434699ff390c66ff01e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45861 } } peers { permanent_uuid: "4d6973e849ab43478a43d575c20ec7ab" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 } } peers { permanent_uuid: "128e950eab08434699ff390c66ff01e2" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } }
I20260811 05:18:12.788846 11638 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2d4d27b157849a19a0d8302b5bd9cb2. Latest consensus state: current_term: 1 leader_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45861 } } peers { permanent_uuid: "4d6973e849ab43478a43d575c20ec7ab" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 } } peers { permanent_uuid: "128e950eab08434699ff390c66ff01e2" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } } }
I20260811 05:18:12.788988 11638 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2 [sys.catalog]: This master's current role is: LEADER
I20260811 05:18:12.789667 11571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 128e950eab08434699ff390c66ff01e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 05:18:12.793146 11526 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "128e950eab08434699ff390c66ff01e2"
I20260811 05:18:12.793402 11526 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 128e950eab08434699ff390c66ff01e2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 05:18:12.794864 11453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.779568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50156 (local address 127.7.67.189:44157)
0811 05:18:12.779783 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:12.779789 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:12.781360 (+ 1571us) server_negotiation.cc:407] Connection header received
0811 05:18:12.781561 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:12.781565 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:12.781619 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:12.781709 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:12.782684 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.783411 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.787229 (+ 3818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.787546 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.787651 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:12.790195 (+ 2544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:12.790201 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:12.790205 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:12.790235 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:12.792108 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.792931 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.792934 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.792936 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.793030 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.793453 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.793457 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.793459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.793688 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:12.793811 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:12.794534 (+ 723us) server_negotiation.cc:299] Negotiation successful
0811 05:18:12.794664 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260811 05:18:12.794991 11526 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 128e950eab08434699ff390c66ff01e2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2d4d27b157849a19a0d8302b5bd9cb2 in term 1.
I20260811 05:18:12.796733 11611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d6973e849ab43478a43d575c20ec7ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45861 } } peers { permanent_uuid: "4d6973e849ab43478a43d575c20ec7ab" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 } } peers { permanent_uuid: "128e950eab08434699ff390c66ff01e2" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } } }
I20260811 05:18:12.796881 11611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d6973e849ab43478a43d575c20ec7ab [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:18:12.789029 11347 master_runner.cc:186] Error getting master registration for 127.7.67.188:39185: OK, instance_id { permanent_uuid: "128e950eab08434699ff390c66ff01e2" instance_seqno: 1786425492715620 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 05:18:12.797981 11347 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 05:18:12.792676 11645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 128e950eab08434699ff390c66ff01e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45861 } } peers { permanent_uuid: "4d6973e849ab43478a43d575c20ec7ab" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 } } peers { permanent_uuid: "128e950eab08434699ff390c66ff01e2" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } } }
I20260811 05:18:12.798087 11645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 128e950eab08434699ff390c66ff01e2 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 05:18:12.800398 11666 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 128e950eab08434699ff390c66ff01e2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:18:12.800477 11666 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:18:12.801239 11647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 05:18:12.801450 11341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.784632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47950 (local address 127.7.67.190:45861)
0811 05:18:12.784966 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:12.784972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:12.789699 (+ 4727us) server_negotiation.cc:407] Connection header received
0811 05:18:12.789909 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:12.789913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:12.789965 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:12.790027 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:12.791056 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.791881 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.793073 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.793394 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.793485 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:12.795360 (+ 1875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:12.795365 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:12.795368 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:12.795399 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:12.797910 (+ 2511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.799797 (+ 1887us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.799801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.799803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.799911 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.800277 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.800281 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.800282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.800570 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:12.800711 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:12.801151 (+ 440us) server_negotiation.cc:299] Negotiation successful
0811 05:18:12.801289 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250}
I20260811 05:18:12.802348 11647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 05:18:12.803190 11577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:12.784223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43608 (local address 127.7.67.188:39185)
0811 05:18:12.784376 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:12.784380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:12.784393 (+ 13us) server_negotiation.cc:407] Connection header received
0811 05:18:12.784428 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:12.784432 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:12.784476 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:12.784598 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:12.785630 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.786169 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.796101 (+ 9932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:12.796498 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:12.796687 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:12.798974 (+ 2287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 05:18:12.798978 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 05:18:12.798980 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 05:18:12.799005 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 05:18:12.800901 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.801473 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.801476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.801478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.801568 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 05:18:12.802122 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 05:18:12.802128 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 05:18:12.802132 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 05:18:12.802376 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 05:18:12.802504 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:12.802933 (+ 429us) server_negotiation.cc:299] Negotiation successful
0811 05:18:12.803048 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 05:18:12.807583 11526 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 128e950eab08434699ff390c66ff01e2 [term 1 FOLLOWER]: Refusing update from remote peer f2d4d27b157849a19a0d8302b5bd9cb2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:18:12.808372 11405 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4d6973e849ab43478a43d575c20ec7ab [term 1 FOLLOWER]: Refusing update from remote peer f2d4d27b157849a19a0d8302b5bd9cb2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 05:18:12.808736 11638 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "128e950eab08434699ff390c66ff01e2" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 05:18:12.809880 11638 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d6973e849ab43478a43d575c20ec7ab" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 05:18:12.811659 11635 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4d6973e849ab43478a43d575c20ec7ab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 05:18:12.811745 11635 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 05:18:12.815100 11611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d6973e849ab43478a43d575c20ec7ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2d4d27b157849a19a0d8302b5bd9cb2. Latest consensus state: current_term: 1 leader_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45861 } } peers { permanent_uuid: "4d6973e849ab43478a43d575c20ec7ab" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 } } peers { permanent_uuid: "128e950eab08434699ff390c66ff01e2" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } } }
I20260811 05:18:12.815219 11611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d6973e849ab43478a43d575c20ec7ab [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:18:12.815366 11645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 128e950eab08434699ff390c66ff01e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2d4d27b157849a19a0d8302b5bd9cb2. Latest consensus state: current_term: 1 leader_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45861 } } peers { permanent_uuid: "4d6973e849ab43478a43d575c20ec7ab" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 } } peers { permanent_uuid: "128e950eab08434699ff390c66ff01e2" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } } }
I20260811 05:18:12.815474 11645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 128e950eab08434699ff390c66ff01e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:18:12.821785 11645 sys_catalog.cc:455] T 00000000000000000000000000000000 P 128e950eab08434699ff390c66ff01e2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45861 } } peers { permanent_uuid: "4d6973e849ab43478a43d575c20ec7ab" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 } } peers { permanent_uuid: "128e950eab08434699ff390c66ff01e2" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } } }
I20260811 05:18:12.821909 11645 sys_catalog.cc:458] T 00000000000000000000000000000000 P 128e950eab08434699ff390c66ff01e2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:18:12.822052 11611 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4d6973e849ab43478a43d575c20ec7ab [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45861 } } peers { permanent_uuid: "4d6973e849ab43478a43d575c20ec7ab" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 } } peers { permanent_uuid: "128e950eab08434699ff390c66ff01e2" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } } }
I20260811 05:18:12.822162 11611 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4d6973e849ab43478a43d575c20ec7ab [sys.catalog]: This master's current role is: FOLLOWER
I20260811 05:18:12.822232 11681 mvcc.cc:205] Tried to move back new op lower bound from 7317198818533158912 to 7317198818436182016. Current Snapshot: MvccSnapshot[applied={T|T < 7317198818533158912}]
I20260811 05:18:12.823069 11638 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45861 } } peers { permanent_uuid: "4d6973e849ab43478a43d575c20ec7ab" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 } } peers { permanent_uuid: "128e950eab08434699ff390c66ff01e2" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } } }
I20260811 05:18:12.823196 11638 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2 [sys.catalog]: This master's current role is: LEADER
I20260811 05:18:12.823315 11638 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2d4d27b157849a19a0d8302b5bd9cb2" member_type: VOTER last_known_addr { host: "127.7.67.190" port: 45861 } } peers { permanent_uuid: "4d6973e849ab43478a43d575c20ec7ab" member_type: VOTER last_known_addr { host: "127.7.67.189" port: 44157 } } peers { permanent_uuid: "128e950eab08434699ff390c66ff01e2" member_type: VOTER last_known_addr { host: "127.7.67.188" port: 39185 } } }
I20260811 05:18:12.823405 11638 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2 [sys.catalog]: This master's current role is: LEADER
I20260811 05:18:12.823545 11676 mvcc.cc:205] Tried to move back new op lower bound from 7317198818533158912 to 7317198818436182016. Current Snapshot: MvccSnapshot[applied={T|T < 7317198818533158912}]
I20260811 05:18:12.835451 11647 catalog_manager.cc:1347] Generated new cluster ID: ef71da26c4614ab093be14402d50180f
I20260811 05:18:12.835542 11647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 05:18:12.863791 11647 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 05:18:12.865222 11647 catalog_manager.cc:1504] Loading token signing keys...
I20260811 05:18:12.894208 11647 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2d4d27b157849a19a0d8302b5bd9cb2: Generated new TSK 0
I20260811 05:18:12.895251 11647 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 05:18:12.931672 11453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:12.929257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50170 (local address 127.7.67.189:44157)
0811 05:18:12.929399 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:12.929405 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:12.930545 (+ 1140us) server_negotiation.cc:407] Connection header received
0811 05:18:12.930746 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:12.930751 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:12.930800 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:12.930868 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:12.931525 (+ 657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50170: BlockingRecv error: recv got EOF from 127.0.0.1:50170 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 05:18:12.933112 11577 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:12.930011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43616 (local address 127.7.67.188:39185)
0811 05:18:12.930125 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:12.930130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:12.932200 (+ 2070us) server_negotiation.cc:407] Connection header received
0811 05:18:12.932344 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:12.932347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:12.932396 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:12.932455 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:12.932982 (+ 527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43616: BlockingRecv error: recv got EOF from 127.0.0.1:43616 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260811 05:18:12.934924 11341 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:12.933766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47966 (local address 127.7.67.190:45861)
0811 05:18:12.933905 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:12.933911 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:12.934115 (+ 204us) server_negotiation.cc:407] Connection header received
0811 05:18:12.934234 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:12.934240 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:12.934298 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:12.934399 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:12.934798 (+ 399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47966: BlockingRecv error: recv got EOF from 127.0.0.1:47966 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260811 05:18:12.942608 11453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:12.941078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50184 (local address 127.7.67.189:44157)
0811 05:18:12.941181 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:12.941185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:12.941618 (+ 433us) server_negotiation.cc:407] Connection header received
0811 05:18:12.941868 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:12.941871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:12.941949 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:12.942004 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:12.942508 (+ 504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50184: BlockingRecv error: recv got EOF from 127.0.0.1:50184 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 05:18:12.947738 11453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:12.946162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50192 (local address 127.7.67.189:44157)
0811 05:18:12.946264 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:12.946270 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:12.946796 (+ 526us) server_negotiation.cc:407] Connection header received
0811 05:18:12.947022 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:12.947026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:12.947078 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:12.947133 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:12.947633 (+ 500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50192: BlockingRecv error: recv got EOF from 127.0.0.1:50192 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260811 05:18:12.953627 11453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:12.951918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50194 (local address 127.7.67.189:44157)
0811 05:18:12.952030 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:12.952035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:12.952654 (+ 619us) server_negotiation.cc:407] Connection header received
0811 05:18:12.952873 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:12.952877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:12.952930 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:12.952986 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:12.953524 (+ 538us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50194: BlockingRecv error: recv got EOF from 127.0.0.1:50194 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260811 05:18:12.961966 11453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:12.960212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50200 (local address 127.7.67.189:44157)
0811 05:18:12.960367 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:12.960371 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:12.960958 (+ 587us) server_negotiation.cc:407] Connection header received
0811 05:18:12.961159 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:12.961163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:12.961231 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:12.961288 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:12.961841 (+ 553us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50200: BlockingRecv error: recv got EOF from 127.0.0.1:50200 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260811 05:18:12.977993 11453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:12.972472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50210 (local address 127.7.67.189:44157)
0811 05:18:12.972660 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:12.972665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:12.976850 (+ 4185us) server_negotiation.cc:407] Connection header received
0811 05:18:12.977096 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:12.977100 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:12.977161 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:12.977221 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:12.977864 (+ 643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50210: BlockingRecv error: recv got EOF from 127.0.0.1:50210 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
W20260811 05:18:12.998629 11453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:12.996693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50224 (local address 127.7.67.189:44157)
0811 05:18:12.996811 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:12.996816 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:12.997470 (+ 654us) server_negotiation.cc:407] Connection header received
0811 05:18:12.997720 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:12.997724 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:12.997796 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:12.997863 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:12.998494 (+ 631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50224: BlockingRecv error: recv got EOF from 127.0.0.1:50224 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260811 05:18:13.035431 11453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:13.034215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50232 (local address 127.7.67.189:44157)
0811 05:18:13.034395 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:13.034401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:13.034417 (+ 16us) server_negotiation.cc:407] Connection header received
0811 05:18:13.034478 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:13.034482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:13.034536 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:13.034624 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:13.035301 (+ 677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50232: BlockingRecv error: recv got EOF from 127.0.0.1:50232 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 05:18:13.103240 11453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:13.102040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50234 (local address 127.7.67.189:44157)
0811 05:18:13.102222 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:13.102227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:13.102372 (+ 145us) server_negotiation.cc:407] Connection header received
0811 05:18:13.102489 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:13.102492 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:13.102543 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:13.102621 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:13.103091 (+ 470us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50234: BlockingRecv error: recv got EOF from 127.0.0.1:50234 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 05:18:13.234985 11453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:13.233776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50238 (local address 127.7.67.189:44157)
0811 05:18:13.233938 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:13.233943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:13.234100 (+ 157us) server_negotiation.cc:407] Connection header received
0811 05:18:13.234219 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:13.234221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:13.234282 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:13.234373 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:13.234828 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50238: BlockingRecv error: recv got EOF from 127.0.0.1:50238 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 05:18:13.494704 11453 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 05:18:13.493373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50248 (local address 127.7.67.189:44157)
0811 05:18:13.493536 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:13.493541 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:13.493683 (+ 142us) server_negotiation.cc:407] Connection header received
0811 05:18:13.493901 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:13.493910 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:13.493965 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:13.494064 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 05:18:13.494581 (+ 517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50248: BlockingRecv error: recv got EOF from 127.0.0.1:50248 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 05:18:13.811015 11666 catalog_manager.cc:1259] Loaded cluster ID: ef71da26c4614ab093be14402d50180f
I20260811 05:18:13.811121 11666 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 128e950eab08434699ff390c66ff01e2: loading cluster ID for follower catalog manager: success
I20260811 05:18:13.812938 11635 catalog_manager.cc:1259] Loaded cluster ID: ef71da26c4614ab093be14402d50180f
I20260811 05:18:13.813004 11635 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4d6973e849ab43478a43d575c20ec7ab: loading cluster ID for follower catalog manager: success
I20260811 05:18:13.813035 11666 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 128e950eab08434699ff390c66ff01e2: acquiring CA information for follower catalog manager: success
I20260811 05:18:13.814265 11666 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 128e950eab08434699ff390c66ff01e2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 05:18:13.814595 11635 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4d6973e849ab43478a43d575c20ec7ab: acquiring CA information for follower catalog manager: success
I20260811 05:18:13.815611 11635 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4d6973e849ab43478a43d575c20ec7ab: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 05:18:14.013051 11793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:14.009120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50262 (local address 127.7.67.189:44157)
0811 05:18:14.009387 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:14.009391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:14.009519 (+ 128us) server_negotiation.cc:407] Connection header received
0811 05:18:14.009665 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:14.009667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:14.009804 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:14.009954 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:18:14.010198 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:14.010748 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:14.011558 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:14.011775 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:14.011849 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:14.012145 (+ 296us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:18:14.012640 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:18:14.012732 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:14.012798 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 05:18:14.012845 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":71,"threads_started":1}
I20260811 05:18:14.019950 11801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:14.016094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43624 (local address 127.7.67.188:39185)
0811 05:18:14.016307 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:14.016310 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:14.016324 (+ 14us) server_negotiation.cc:407] Connection header received
0811 05:18:14.016405 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:14.016407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:14.016518 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:14.016671 (+ 153us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:18:14.016866 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:14.017357 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:14.018124 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:14.018340 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:14.018422 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:14.018677 (+ 255us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:18:14.019412 (+ 735us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:18:14.019517 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:14.019580 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 05:18:14.019628 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":68,"threads_started":1}
I20260811 05:18:14.027360 11801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:14.023057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43638 (local address 127.7.67.188:39185)
0811 05:18:14.023265 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:14.023269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:14.023654 (+ 385us) server_negotiation.cc:407] Connection header received
0811 05:18:14.023782 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:14.023785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:14.023839 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:14.023924 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:18:14.024319 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:14.024880 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:14.026143 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:14.026466 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:14.026557 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:14.026919 (+ 362us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:18:14.027009 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:18:14.027091 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:14.027183 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 05:18:14.027235 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140}
I20260811 05:18:14.027526 11793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:14.022941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50264 (local address 127.7.67.189:44157)
0811 05:18:14.023159 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:14.023166 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:14.023534 (+ 368us) server_negotiation.cc:407] Connection header received
0811 05:18:14.023655 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:14.023659 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:14.023712 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:14.023783 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:18:14.024227 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:14.025037 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:14.026231 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:14.026516 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:14.026615 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:14.026991 (+ 376us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:18:14.027064 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:18:14.027135 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:14.027322 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 05:18:14.027383 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133}
I20260811 05:18:14.027971 11810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 05:18:14.023088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47974 (local address 127.7.67.190:45861)
0811 05:18:14.023370 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0811 05:18:14.023374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 05:18:14.023389 (+ 15us) server_negotiation.cc:407] Connection header received
0811 05:18:14.023571 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 05:18:14.023578 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 05:18:14.023775 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 05:18:14.023899 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 05:18:14.024351 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:14.025153 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:14.026024 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 05:18:14.026387 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 05:18:14.026502 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 05:18:14.026839 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 05:18:14.027180 (+ 341us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 05:18:14.027268 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 05:18:14.027704 (+ 436us) server_negotiation.cc:299] Negotiation successful
0811 05:18:14.027771 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":89,"threads_started":1}
I20260811 05:18:14.035574 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 11240
I20260811 05:18:14.052572 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 11347
I20260811 05:18:14.064464 7438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktf25Hl/build/debug/bin/kudu with pid 11461
2026-08-11T05:18:14Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3409 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6881 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (303399 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST